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EXECUTIVE  SUMMARY 


This  report  documents  the  findings  of  two  studies  of  air  traffic  controller  reaction  to  the 
Precision  Runway  Monitor  (PRM).  The  studies  were  conducted  in  order  to  determine  the  effects  of 
key  variables  on  controller  reaction  time  and  to  determine  controller  opinion  on  the  acceptability  of 
the  system. 

BACKGROUND 

The  primary  purpose  of  radar  monitoring  is  to  insure  safe  separation  of  aircraft  arriving  on 
the  parallel  approach  courses.  This  separation  is  compromised  if  an  aircraft  deviates  off  course 
towards  an  aircraft  on  the  adjacent  approach.  The  PRM  was  designed  to  facilitate  the  monitoring 
process. 

The  PRM  is  a  new  radar  monitoring  system  which  consists  of  radar  which  has  higher  accuracy 
and  a  shorter  update  interval  than  the  current  system.  The  PRM  radar  is  accompanied  by  a  high- 
resolution,  color  display  which  provides  automated  visual  and  vocal  warnings  to  alert  controllers  of 
impending  and  actual  penetration  of  a  “No  Transgression  Zone"  (NTZ)  between  parallel  runways. 

STUDY  METHOD 

Fifty  air  traffic  controllers  participated  in  Study  I,  which  examined  the  use  of  the  PRM  when 
the  runway  separation  was  both  3,400  ft  and  4,300  ft.  Ten  air  traffic  controllers  participated  in 
Study  II,  which  examined  the  use  of  the  PRM  when  the  runway  separation  was  3,000  ft.  Both  studies 
were  conducted  at  Memphis  International  Airport 

The  testing  consisted  of  the  presentation  of  real-time  simulated  approach  blunders  and 
measurement  of  controller  reaction  time  in  directing  endangered  aircraft  out  of  the  path  of  an  aircraft 
deviating  from  the  adjacent  parallel  approach.  Controller  reaction  was  measured  in  terms  of  Alert 
Response  Time  (ART).  Measurement  of  ART  begins  when  the  Caution  Alert  is  given  and  ends  when 
the  controller  begins  to  speak  the  breakout  instruction  to  the  endangered  aircraft 

Independent  variables  explored  included:  sensor  update  interval,  runway  separation,  deviation 
angle,  deviation  range,  flight  path  condition,  approach  blunder  type,  and  controller  experience  level. 
In  addition,  controller  opinions  of  the  PRM  were  surveyed. 

OVERALL  RESULTS 

Results  of  Study  I  indicated  that  with  3,400-ft  runway  separation,  controller  reaction  time  was 
not  significantly  affected  by  sensor  update  interval  in  the  case  of  1.0-s  vs  2.4-s  sensor  update  interval. 
However,  significant  delays  in  reaction  time  did  occur  when  the  update  interval  was  4.8  s.  While 
controller  reaction  time  was  not  significantly  affected  in  the  case  of  1.0-s  vs  2.4-s  sensor  update 
interval,  the  faster  sensor  update  interval  did  demonstrate  some  advantages.  The  faster  sensor  update 
interval  provided  increased  advanced  warning  that  translates  into  a  greater  missed  distance  between 
aircraft  The  faster  sensor  update  interval  resulted  in  a  lower  false  breakout  rate  that  translates  into 
less  arrival  delays. 

Deviations  of  30  deg  were  found  to  result  in  quicker  reaction  times  than  the  lesser  deviations 
of  15  deg.  Deviations  occurring  far  from  the  runway  threshold  resulted  in  slower  reaction  times  in 
comparison  to  deviations  occurring  near  the  runway  threshold.  No  major  effects  were  attributable  to 
flight  path  condition  and  approach  blunder  type.  Experienced  Monitor  Controllers  demonstrated 


iii 


slower  reaction  times  than  controllers  who  did  not  have  Monitor  Controller  experience.  Regarding 
the  4.8-s  sensor  update  interval,  it  was  found  that  controller  reaction  was  significantly  faster  when  the 
runway  separation  was  4,300  ft  as  compared  to  3,400  ft. 

Results  of  Study  II  indicated  that  with  3,000-ft  runway  separation,  whether  the  sensor  update 
interval  was  1.0-s  or  2.4-s,  controller  reaction  time  remained  basically  the  same.  The  reaction  times 
did  not  differ  significantly  from  those  of  Study  I  in  which  the  same  update  intervals  were  used  but  the 
runway  separation  was  3,400  ft  However,  due  to  the  closeness  of  the  runways  in  the  case  of  3,000-ft 
separation,  total  navigational  system  error  (TNSE)  caused  a  number  of  aircraft  to  near  or  enter  the 
NTZ.  As  a  result,  nuisance  alerts  occurred  and  the  probability  of  false  breakouts  increased. 

Controllers'  opinions  of  the  system  indicated  a  high  level  of  acceptance.  The  vast  majority  of 
controllers  felt  that  the  system  could  be  used  to  safely  conduct  independent  approaches  to  runways 
with  3,400-ft  spacing  if  a  1.0-s  or  2.4-s  sensor  update  interval  were  used.  Controllers  were  divided  on 
their  opinion  of  the  safety  of  use  of  the  system  when  runways  are  separated  by  3,000  ft.  The  major 
concern  was  the  effect  of  TNSE  mentioned  above. 

RECOMMENDATIONS 

Recommendations  are  made  regarding  ways  in  which  to  enhance  the  realism  of  the 
simulation,  if  future  studies  were  to  be  conducted.  The  inclusion  of  background  noise  appropriate  to 
the  work-setting  is  recommended.  Enhancements  to  the  recorded  audio  are  recommended. 

Recommendations  arc  made  for  further  testing  of  the  case  of  3,000-ft  runway  separation.  It 
is  also  recommended  that  the  look-ahead  time  for  the  Caution  Alert  be  further  explored. 
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L  INTRODUCTION 


Increased  air  travel  in  recent  years  has  resulted  in  a  steady  increase  in  the  number  and 
duration  of  flight  delays.  Airports  have  not  been  able  to  expa^a  to  keep  pace  with  traffic  growth.  In 
an  attempt  to  increase  airport  capacity,  'he  Federal  Aviation  Administration  (FAA)  has  taken  a  variety 
of  measures  to  increase  airport  capacity.  Measures  have  included  revisions  to  air  traffic  control 
procedures,  addition  of  landing  systems,  taxiways,  and  runways;  and  application  of  new  technology. 

The  Precision  Runway  Monitor  (PRM)  is  a  prime  example  of  the  use  of  new  technology  to 
address  the  need  for  increased  capacity.  The  PRM  is  an  advanced  radar  monitoring  system  designed 
to  increase  utilization  of  closely-spaced,  multiple,  parallel  runways  during  adverse  weather  conditions. 
The  main  objective  of  the  PRM  Program  is  to  develop  an  improved  radar  and  the  associated 
procedures  necessary  to  lower  the  minimum  required  spacing  between  parallel  runways  for 
independent  Instrument  Landing  System  (ILS)  operations. 

Two  versions  of  the  new  radar  and  accompanying  display  system  for  the  air  traffic  controllers 
were  developed  by  the  FAA  through  contract  with  MIT  Lincoln  Laboratory  and  MSI  Services,  Inc. 
Prototypes  of  the  systems  were  installed  at  two  airports.  The  Lincoln  Laboratory  prototype,  the 
Mode  S  Sensor,  was  installed  at  Memphis,  TN,  International  Airport.  The  MSI  and  Bendix 
Corporation  prototype,  the  E-Scan  Sensor,  was  installed  at  Raleigh-Durham,  NC,  International 
Airport. 

In  both  prototype  systems,  the  new  monitoring  equipment  consists  of  radar  and  displays.  The 
radar  portion  of  the  prototypes  were  not  functionally  equivalent.  A  radar  having  electronically 
scanned  antenna  capable  of  half-second  update  intervals  was  installed  at  Raleigh,  while  a  Mode  S- 
based  sensor  with  a  mechanically  rotating  “back-to-back”  antenna  capable  of  2.4-s  sensor  update 
intervals  was  installed  at  Memphis. 

The  display  portion  of  both  prototype  systems  were  functionally  equivalent  The  display 
presents  simultaneous  approaches  to  parallel  runways  on  high-resolution,  color,  and  expanded  screen. 
As  decision  aids  for  the  air  traffic  controllers,  two-level,  color-coded,  visual  and  audible  alerts  were 
included. 

A  human  factors  approach  was  used  in  the  design  of  the  display.  Potential  users  of  the 
display  played  a  key  role  in  its  development  A  group  of  experienced  air  traffic  controllers  from 
Memphis  and  Raleigh-Durham  Air  Traffic  Control  (ATC)  Towers  worked  as  part  of  the  design  team. 
This  group  is  referred  to  as  the  Core  Group  of  Controllers.  Together  with  hardware  and  software 
engineers  and  a  human  factors  psychologist,  the  various  elements  of  the  display  were  identified  and 
implemented.  The  PRM  Display  is  briefly  discussed  in  Section  3. 1.2.1.  For  a  detailed  description  of 
the  elements  included  in  the  display  the  reader  is  referred  to  “User’s  Guide  to  the  MIT  Lincoln 
Laboratory  Precision  Runway  Monitor  System”  [1J, 
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1.1  EVALUATION  OF  SYSTEM  EFFECTIVENESS 

In  order  to  evaluate  the  effectiveness  of  the  PRM,  a  myriad  of  data  were  collected.  This  was 
done  in  order  to  address  concerns  inherent  in  making  a  decision  regarding 

(1)  whether  or  not  the  current  standard  for  runway  separation  of  4,300  ft  can  be  reduced  to 
3,400  ft  or  less  when  tire  PRM  is  used,  and 

(2)  whether  or  not  a  1.0-s  or  2.4-s  sensor  update  interval  is  required. 

The  data  collected  at  Raleigh-Durham  include  pilot  response  times  gathered  through  live 
aircraft  testing  and  simulation.  They  also  include  air  traffic  controller  response  times  in  using  the 
PRM  to  monitor  simulated,  independent  parallel  approaches.  The  data  collected  at  Memphis  include 
pilot  response  times  in  live  aircraft  testing  and  air  traffic  controller  response  times  in  using  the  PRM 
to  monitor  simulated,  independent  parallel  approaches.  Memphis  data  also  include  measurements  of 
total  navigational  system  error  (TNSE)  [2].  Data  on  pilot  response  times  were  collected  through 
simulator  studies  conducted  by  the  FAA  Standards  Development  Branch  [3, 4]. 

Much  of  the  data  collected  were  used  as  input  to  a  risk  assessment  model  developed  by 
Lincoln  Laboratory  [5,  6].  The  model  provides  a  means  for  quantifying  the  risk  inherent  in 
conducting  independent  approaches  to  closely-spaced  parallel  runways.  It  is  a  useful  tool  for 
evaluating  changes  in  risk  which  result  from  varying  the  values  of  relevant  variables. 

Partial  findings  regarding  the  effectiveness  of  the  PRM  were  documented  in  ‘The  Precision 
Runway  Monitor  Demonstration  Report”  [5],  published  in  February  1991.  The  report  documents 
key  results  obtained  from  the  PRM  equipment  demonstrations  and  studies  at  the  Memphis  and 
Raleigh-Durham  airports.  It  includes  the  results  of  application  of  the  risk  assessment  model.  The 
report  recommends  a  new  standard  for  the  minimum  allowable  runway  separation  for  independent 
approaches  to  parallel  runways.  It  recommends  that: 

the  FAA  issue  a  national  standard  for  runway  spacing  of  3,400  ft,  provided  the 
approaches  can  be  monitored  by  displays  equivalent  to  those  used  in  the 
demonstration,  driven  by  a  radar  accurate  to  within  1  milliradian  with  an  update 
interval  of  2.4  s  or  less.  With  this  equipment,  the  risk  of  a  blunder  resulting  in  less 
than  500  ft  of  separation  when  two  aircraft  are  on  parallel  approach  in  IFR  conditions, 
is  less  than  one  in  250  million  instrument  approaches.  This  recommendation  is 
contingent  on  successful  deployment  of  a  satisfactory  surveillance  and  display 
system.  A  familiarization  program  to  ensure  that  all  pilots  understand  their 
responsibilities  during  a  closely-spaced  parallel  approach  will  also  be  required.  An 
off-centerline  obstruction  evaluation  will  be  conducted  at  all  airports  where  the  PRM 
is  to  be  installed. 
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1 .2  SCOPE  OF  THIS  REPORT 


The  current  report  documents  the  findings  of  one  portion  of  the  data  collected  in  the  data 
collection  effort,  briefly  described  above.  The  report  focuses  on  the  reactions  of  the  air  traffic 
controllers  who  participated  in  two  PRM  studies  conducted  at  Memphis  International  Airport 

Study  I  was  conducted  from  February  5  to  August  3, 1990,  and  examined  the  effectiveness  of 
the  PRM  in  aiding  controllers  in  monitoring  independent  approaches  to  parallel  runways  separated 
by  3,400  ft  and  4,300  ft  As  a  follow-on  study,  Study  n  was  conducted  from  November  26  to 
December  12,  1990,  and  examined  independent  approaches  to  parallel  runways  separated  by 
3,000  ft 

Controller  reaction  results  from  Memphis  appeared  in  “The  Precision  Runway  Monitor 
Demonstration  Report”  [5].  However,  they  were  presented  at  a  summary  level.  Report  deadlines 
prevented,  in  some  cases,  inclusion  of  a  portion  of  the  controller  reaction  data  collected  in  the 
Memphis  Studies.  The  purpose  of  the  current  report  is  to  provide  a  detailed  analysis  of  those 
controller  reactions  and  to  discuss  how  those  reactions  were  affected  by  key  variables.  Although  a 
risk  assessment  model  may  tell  us  that  an  operation  is  “safe,”  the  analysis  of  the  effect  of  each 
variable  tells  us  how  reactions  varied  within  that  safety  window. 

1.3  ORGANIZATION  OF  THIS  REPORT 

Section  1  includes  a  discussion  of  the  capacity  problem  which  lead  to  the  development  of  the 
PRM.  It  also  provides  background  information  on  independent  and  dependent  approaches  and  their 
effect  on  capacity.  The  role  of  the  Monitor  Controller,  who  is  the  user  evaluating  this  system,  is 
discussed. 

Section  2  explains  the  experimental  design  used  in  collecting  the  controller  data  in  both 
Memphis  Studies.  All  independent  and  dependent  variables  are  defined.  The  stimulus  for  eliciting 
the  response  of  the  controllers  is  also  discussed,  i.e.,  simulated  approach  blunders. 

Section  3  details  the  methods  and  exact  procedures  followed,  as  well  as  the  procedures  for 
data  collection.  Section  4  discusses  the  results  of  Study  I.  Section  5  discusses  the  results  of  Study  n. 
Section  6  gives  conclusions  and  recommendations  for  future  study. 

1.4  BACKGROUND 

1.4.1  The  Capacity  Problem  and  the  Development  of  the  PRM 

Airport  capacity  is  significantly  enhanced  when  independent  approaches  to  parallel  runways 
are  available.  Current  FAA  procedures  permit  independent  approaches  in  instrument  meteorological 
conditions  (IMQ  when  the  parallel  runways  are  spaced  at  least  4,300  ft  apart.  At  airports  that  have 
parallel  runways  separated  by  less  than  4,300  ft  or  have  converging  runways,  arriving  aircraft  must  be 
dependency  sequenced,  a  procedure  that  reduces  the  arrival  rate. 

The  need  for  greater  airport  capacity  has  led  to  intense  interest  in  the  use  of  new  technologies 
that  can  support  independent  parallel  IMC  approaches  to  runways  having  spacings  as  little  as  3,000  ft. 
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This  interest  resulted  in  several  new  FAA  initiatives,  including  an  evaluation  of  the  applicability  of 
sensors  capable  of  high  accuracy  and  rapid  update.  The  sensors  were  installed  at  Memphis  and 
Raleigh  in  order  to  assess  the  feasibility  of  using  them  for  parallel  runway  approach  monitoring. 

1.4.2  Simultaneous  Instrument  Landing  System  Procedure 

Section  1.4.2  through  1.4.6  are  provided  in  order  to  help  the  reader  understand  current 
approach  procedures  and  the  potential  benefits  of  the  PRM.  These  sections  describe  how  pilots 
navigate  and  how  controllers  direct  them  to  land  in  bad  weather.  These  techniques  are  first  described 
for  a  single  runway,  and  then  for  multiple  runways  at  the  same  airport  Next  existing  limitations  to 
full  runway  utilization  are  explained,  followed  by  a  discussion  of  how  the  limitations  might  be 
avoided  with  the  PRM. 

1.4 .3  Instrument-Approach  Procedures 

During  IMC,  a  variety  of  procedures  have  been  developed  to  guide  appropriately  equipped 
aircraft  safely  to  the  vicinity  of  the  runway.  The  most  precise  procedure  in  common  use  is  the 
Instrument  Landing  System  (ILS).  Radio-navigation  signals  identify  a  precise  flight  path,  laterally 
with  the  localizer,  and  vertically  with  the  glide  slope.  The  signals  are  displayed  to  the  flight  crew  on 
an  instrument  that  indicates  the  location  of  the  flight  path  relative  to  current  aircraft  position. 

At  busy  airports,  air  traffic  controllers  use  radar  to  direct  the  aircraft  to  intercept  the  localizer 
5  to  15  nautical  miles  (nmi)  from  the  runway  threshold.  Aircraft  reach  this  intercept  one  at  a  time, 
separated  by  at  least  3  nmi  from  the  aircraft  ahead.  The  aircraft  then  follow  the  localizer  signal  at 
constant  altitude,  and  begin  descending  when  the  glide  slope  is  intercepted.  When  an  aircraft  reaches 
the  missed  approach  point  (MAP),  typically  0.5  nmi  from  and  200  ft  above  the  runway  threshold,  the 
flight  crew  must  be  able  to  see  the  runway  environment  and  complete  the  landing  visually.  If  they 
are  unable  to  do  so,  they  must  reject  the  landing  and  follow  a  missed  approach  procedure. 

1.4.4  Parallel  Runway  Simultaneous  ILS  Approaches 

The  procedures  for  airports  with  multiple  parallel  runways  are  similar,  with  added  safeguards 
to  ensure  that  an  aircraft  approaching  one  runway  is  safely  separated  from  those  approaching  the 
adjacent  parallel  runway.  The  procedures  are  discussed  in  [7],  and  an  example  of  such  procedures  is 
diagrammed  in  Figure  1-1.  Aircraft  are  directed  to  the  two  final  approach  courses  at  different 
altitudes  separated  by  at  least  1,000  ft.  The  separation  is  necessary  because  the  normally  maintained 
3-nmi  separation  is  lost  as  the  aircraft  fly  toward  their  respective  localizers.  This  1,000-ft  vertical 
separation  is  maintained  until  the  controller  sees  that  each  aircraft  is  stabilized  on  its  parallel  localizer 
course.  Then,  the  aircraft  are  allowed  to  descend  on  their  respective  glide  slopes,  flying  towards  the 
airport  separated  by  the  distance  between  the  runway  centerlines. 

Because  this  separation  is  much  less  than  the  3  nmi  normally  maintained,  the  two  aircraft  are 
monitored  on  radar  starting  when  the  1,000-ft  altitude  buffer  is  lost  as  the  higher  aircraft  starts  down 
the  glide  slope.  Two  controllers,  called  Monitor  Controllers,  observe  the  parallel  approaches  and 
ensure  that  if  an  aircraft  blunders  from  the  normal  operating  zone  (NOZ)  into  a  2,000-ft  NTZ,  as 
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shown  in  Figure  1*1,  any  endangered  aircraft  on  the  other  approach  are  turned  away  in  time  to 
prevent  a  collision.  This  maneuver  on  the  part  of  the  endangered  aircraft  is  termed  a  “breakout," 
because  the  aircraft  is  directed  out  of  the  approach  stream  to  avoid  the  blundering  aircraft.  Two 
controllers  are  necessary  so  that  one  can  attempt  to  turn  the  blundering  (deviating)  aircraft  back  to  its 
localizer  while  the  other  directs  the  breakout  of  the  endangered  aircraft  Typically,  two  separate  radio 
frequencies  are  used,  one  for  each  approach. 

The  2,000-ft  NTZ,  flanked  by  two  equal  NOZs,  is  a  procedural  artifice  which  provides  strong 
guidance  to  the  Monitor  Controller.  Aircraft  are  allowed  to  operate  on  or  near  the  approach  course 
within  the  limits  of  the  NOZ.  If  an  aircraft  strays  into  the  NTZ,  it  is  deemed  to  create  a  hazard  for  an 
aircraft  on  the  adjacent  course.  The  NTZ  width  was  established  to  provide  time  to  resolve  the 
situation  by  redirecting  either  or  both  aircraft  before  a  collision  occurs.  The  2,000-ft  NTZ  width  has 
an  uncertain  origin,  but  pilots  and  controllers  are  confident  that  it  is  appropriate,  based  on  many 
years  of  application  at  the  wider  runway  spacings. 

The  smaller  the  separation  between  the  runway  centerlines,  the  shorter  the  time  that  is 
available  to  correct  a  blunder  once  it  begins.  Parallel  approaches  to  runways  spaced  less  than  4,300  ft 
apart  are  restricted  in  IMC,  in  part  because  the  radar  and  display  systems  currently  available  to  the 
controller  are  sufficiently  imprecise  that  the  blunder  cannot  be  detected  and  corrected  before  the 
aircraft  are  dangerously  close.  For  these  narrower  runway  separations,  a  dependent  or  staggered 
approach  procedure  is  used  to  eliminate  the  risk. 

In  the  dependent  procedure,  controllers  position  aircraft  so  that  there  is  always  at  least  2  nmi 
separating  one  aircraft  from  another  on  the  adjacent  runway.  This  ensures  that  if  an  aircraft  blunders 
toward  the  adjacent  approach,  the  aircraft  will  pass  through  a  gap  and  will  not  encounter  another 
aircraft.  Figure  1-2  diagrams  the  two  situations:  (1)  simultaneous  independent  parallel  approaches, 
when  aircraft  on  one  runway  are  spaced  independently  of  those  on  the  other  runway  (for  runway 
spacings  of  4,300  ft  or  greater),  and  (2)  dependent  parallel  approaches,  when  aircraft  spacing  is 
dependent  on  the  position  of  aircraft  on  the  adjacent  runway  (for  spacings  less  than  4,300  ft).  The 
possibility  of  wake  turbulence  restricts  dependent  parallel  approaches  to  spacings  of  2,500  ft  or 
greater. 
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1.4.5  Arrival  Rate  Penalties  for  Dependent  Approaches 

For  independent  approaches  to  parallel  runways,  the  arrival  rate  is  about  twice  the  single 
runway  rate,  since  the  approaches  to  each  runway  are  independent  and  managed  by  different 
controllers.  But  the  arrival  rate  at  airports  using  dependent  parallel  approaches  is  significantly  less. 
The  required  2-nmi  diagonal  separation  leaves  just  under  a  4-nmi  spacing,  at  minimum,  between 
aircraft  on  the  same  runway.  Independent  approaches  leave  a  3-nmi  or  2.5-nmi  minimum.  The 
different  spacings  yield  a  landing  rate  that  is  about  33%  higher  for  the  independent  case.  In  practice, 
however,  the  coordination  required  to  exactly  produce  the  2-nmi  diagonal  spacing  is  much  more 
complex,  particularly  as  aircraft  typically  arrive  from  many  different  directions  at  a  variety  of  speeds. 
To  insure  that  the  minimum  is  never  violated,  most  controllers  end  up  with  a  spacing  somewhat 
greater  than  the  minimum,  a  reality  which  penalizes  the  dependent  case  even  further. 

Parallel  runway  acceptance  rates  are  thus  significantly  greater  when  independent  approaches 
are  available.  Independence  is  possible  at  700-ft  runway  spacings  during  visual  meteorological 
conditions  (VMQ,  but  only  at  4,300  ft  or  greater  during  IMC.  Numerous  busy  airports  face  serious 
and  costly  arrival  delays  during  IMC  for  this  reason.  Other  airports  with  single  runways  are 
considering  an  additional  parallel  runway  but  are  dissuaded  from  constructing  it  because  land 
acquisition  for  a  4,300-ft  separation  is  either  impossible  or  extremely  expensive. 

The  motivation  for  the  PRM  is  clean  If  improved  radar  surveillance  and  ATC  displays  would 
lead  to  earlier  blunder  warning  to  the  controller,  independent  approaches  could  be  authorized  at 
spacings  less  than  4,300  ft  Arrival  capacity  could  be  increased  immediately  at  some  airports,  and 
construction  at  other  airports  would  yield  greater  benefits.  Figure  1-3  shows  parallel  runway  spacings 
at  some  busy  airports  that  could  benefit  from  a  PRM. 
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Figure  1-3.  Parallel  runway  spacings.  (Airports  with  multiple  runway  pairings, 
such  as  Los  Angeles,  appear  twice.) 
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2 .  EXPERIMENTAL  DESIGN 

Two  controlled  experimental  studies  were  conducted  in  which  simulation  of  approach 
blunders  to  closely-spaced  parallel  runways  were  used  to  measure  controllers'  responses  to  these 
emergency  situations.  Study  I  examined  a  runway  separation  of  3,400  ft.  As  a  result  of  that  study,  a 
second  study,  Study  II,  was  conducted  which  examined  a  runway  separation  of  3,000  ft 

Both  studies  involved  the  measurement  of  controller  reaction  time  in  breaking  out 
endangered  aircraft  when  an  approach  blunder  was  committed  by  an  aircraft  on  the  adjacent  parallel 
approach.  Through  assessment  of  their  responses,  the  effectiveness  of  the  PRM  system,  when  used 
with  runway  separation  which  is  less  than  the  current  standard,  could  be  evaluated.  Opinion  surveys 
were  used  in  order  to  obtain  user  opinion  on  the  safety  of  the  system.  Comments  on  user  preferences 
were  obtained  regarding  information  content  and  presentation  within  the  display. 

In  order  to  evaluate  the  effectiveness  of  the  PRM  and  the  effects  of  key  variables  on 
controller  reaction  time,  controllers  were  presented  with  simulations  of  various  types  of  approach 
blunders,  which  were  systematically  varied  in  each  presentation.  This  allowed  the  exploration  of  the 
effects  of  each  of  the  variables  on  the  response  times  of  the  controllers.  The  major  design  of  the 
studies  was  a  within-subject,  paired-sample  design.  This  means  that  each  controller  was  exposed  to 
one  variable  condition  at  a  time  during  a  sequence  of  approach  blunder  presentations.  The 
controller's  reaction  time  during  one  variable  condition  could  then  be  compared  to  his  or  her  own 
reaction  time  during  another  variable  condition. 

In  studying  the  effects  of  the  variable  “controller  experience  level,”  a  between-subject, 
independent-sample  design  was  used.  This  design  was  used  since  measurements  were  taken  of  two 
groups  of  controllers,  i.e.,  controllers  with  and  controllers  without  Monitor  Controller  experience.  In 
studying  the  effects  of  “runway  separation,”  a  between-subject  independent  sample  design  was  used. 
This  design  was  used  since  two  different  groups  of  controllers  reacted  to  the  condition  of  4.8-s  sensor 
update  interval,  i.e.,  one  group  was  shown  simulations  depicting  3,400-ft  runway  separation  and  the 
other  group  was  shown  4,300- ft  runway  separation. 

Simulation  was  selected  for  use  as  the  means  of  testing,  since  it  provides  several  benefits. 
Approach  blunders  are  a  rare  event  in  actual  air  traffic  operations.  Through  simulation,  the 
controllers  were  exposed  to  the  occurrence  of  approach  blunders,  and,  therefore,  their  reactions  could 
be  observed.  The  use  of  simulation  allowed  for  control  of  the  experimental  variables  which  were 
manipulated.  It  enabled  the  experimenter  to  ensure  that  every  pair  of  controllers  who  participated  in 
the  experiment  witnessed  and  had  the  opportunity  to  react  to  exactly  the  same  stimulus  as  all  other 
controllers  who  participated  in  the  experiment. 

2.1  STUDY! 

2.1.1  Independent  Variables  -  Study  1 

In  each  of  the  monitoring  sessions,  there  were  a  number  of  independent  variables  which  it  was 
believed  might  affect  the  controller’s  responses.  The  variables  were: 

(a)  Sensor  update  interval.  As  the  primary  variable,  this  would  differentiate  between  the 
PRM  alternatives,  and  combined  with  accuracy,  differentiate  the  PRM  from  older 
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sensors.  In  the  simulations,  sensor  update  intervals  of  1.0  s,  2.4  s,  and  4.8  s  were 
tested.  A  4.8-s  interval  was  of  interest  because  monopulse  sensors,  planned  to  replace 
existing  beacon  sensors  around  the  world,  will  rotate  with  this  period  and  provide  the 
same  1-milliradian  accuracy  as  the  PRM.  Although  the  back-to-back  radar  at 
Memphis  was  limited  to  a  2.4-s  interval,  the  simulation  did  not  require  the  radar  itself, 
and  the  displays  were  not  interval-limited. 

(b)  Deviation  angle.  Approach  blunders  were  staged  using  the  worst  case  scenario,  where 
an  aircraft  rolls  smoothly  into  a  standard  rate  turn  and  holds  the  bank  until  a  30-deg 
heading  change  (deviation)  toward  the  adjacent  approach  course  is  achieved. 
Approach  blunders  having  a  15-deg  heading  change  were  also  staged  to  measure  the 
system’s  performance  for  less  severe  approach  blunders. 

(c)  Range  of  the  approach  blunder  from  runway  threshold.  Approach  blunders  were 
staged  both  outside  and  inside  the  outer  marker  and  after  the  missed  approach  point. 
The  difference  is  important  because  of  the  contribution  of  total  navigational  system 
error  (TNSE)  to  the  blunder  starting  conditions  and  the  relative  stability  of  the  aircraft 
at  the  various  spacings.  TNSE  consists  of  the  error  attributed  to  aircraft  flight 
characteristics,  pilot  flying  technique,  and  the  ELS  (Instrument  Landing  System). 

(d)  Flight  Path  Conditions.  Simulation  arrival  periods  modeled  both  calm  conditions  and 
more  turbulent  conditions.  Turbulent  conditions  increase  the  amount  of  TNSE 
during  the  ILS  approach.  It  was  suspected  that  controllers  might  become  more 
tolerant  of  deviations  due  to  turbulence  and  might  delay  their  responses.  This  might 
occur  because  the  beginning  of  an  approach  blunder  could  appear  to  be  a  result  of 
the  increased  TNSE.  In  Study  1,  TNSE  consisted  of  a  sinusoidal  oscillation  off  the 
scripted  flight  path  that  dampened  down  to  zero  amplitude  at  landing.  The  amplitude 
of  the  oscillations  at  a  range  of  IS  nmi  was  one  value  for  “calm  wind  conditions” 
and  a  larger  value  for  “turbulent  conditions.”  This  TNSE,  based  on  preliminary 
radar  observation,  was  reviewed  by  the  Memphis  Core  Group  and  determined  to  be 
representative. 

(e)  Types  of  Approach  Blunders.  Various  types  of  approach  blunders  were  studied. 
Types  consisted  of  “Single,”  “Fast/Slow,”  “Distraction,”  and  “Simultaneous 
Missed  Approach"  Blunders. 

1)  In  a  “Single”  blunder,  one  aircraft  penetrated  the  NTZ  and  endangered  the 
aircraft  on  the  adjacent  approach  path.  Blunders  were  staged  with  both  aircraft 
operating  at  speeds  typical  of  transport  jets. 

2)  A  “Fast/Slow”  blunder  involved  one  aircraft  that  was  fast  (an  air  carrier  traveling 
at  ISO  knots)  and  one  aircraft  that  was  slow  (a  general  aviation  aircraft  traveling  at 
90  knots).  The  fast  aircraft  penetrated  the  NTZ  and  blundered  toward  the  slow 
aircraft  on  the  adjacent  approach  path. 

3)  A  “Distraction”  blunder  was  a  “Single”  blunder  preceded  by  a  distraction.  One 
aircraft  had  an  erratic  flight  path  (one  that  is  not  due  to  turbulent  conditions  but 
presumably  due  to  the  performance  of  the  pilot  or  aircraft).  Then  the  aircraft  on 
the  adjacent  approach  path  deviated  and  penetrated  the  NTZ.  It  was  believed  that 
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a  distraction  might  cause  increased  workload  and  a  resulting  delay  in  controller 
response  time. 

4)  A  “Simultaneous  Missed  Approach”  blunder  involved  two  aircraft,  each  on 
adjacent  approach  paths.  Each  aircraft  performed  a  missed  approach  at  the  same 
time  and  then  blundered  toward  each  other  and  the  NTZ. 

(f)  Controller  Experience  Level.  Approximately  half  of  the  controllers  who  participated 
in  the  study  were  experienced  Monitor  Controllers,  and  half  were  controllers  who  had 
no  previous  experience  as  Monitor  Controllers. 

(g)  Runway  separation.  A  major  finding  of  the  report  was  expected  to  center  on  the 
applicability  of  the  PRM  at  runway  spacings  near  Memphis'  3,400-ft  runway. 
However,  in  order  to  examine  the  effect  at  different  spacings,  a  4,300-ft  simulation 
was  conducted  near  the  latter  half  of  Study  I  to  replace  the  3,400-ft,  4.8-s  sensor 
update  interval  scenarios  on  which  sufficient  data  had  already  been  collected. 
Therefore,  as  of  Week  13  (of  25  weeks  of  testing),  runway  separation  became  an 
independent  variable. 

Since  the  testing  needed  to  be  done  within  the  time  constraint  of  one  week  per  subject  pair,  all 
combinations  of  the  variables  were  not  tested  in  each  type  of  approach  blunder  and  each  sensor 
update  interval.  Therefore,  the  PRM  Working  Group  chose  to  test  only  approach  blunders  of  greatest 
interest.  The  PRM  Working  Group  consisted  of  representatives  from:  the  PRM  Program  Office, 
Flight  Standards,  Air  Traffic  Control,  FAA  Technical  Center,  Aviation  National  Standards  Office, 
Memphis  and  Raleigh-Durham  Air  Traffic  Control  Tower,  MIT  Lincoln  Laboratory,  and  MSI.  The 
combinations  of  variables  selected  for  testing  are  presented  in  Tables  2-1  through  2-3.  Table  2-1  lists 
the  approach  blunders  presented  at  1.0-s  sensor  update  interval,  3,400-ft  runway  separation.  Table 
2-2  lists  the  approach  blunders  presented  at  2.4-s  sensor  update  interval,  3,400-ft  runway  separation. 
The  1.0-s  and  2.4-s,  3,400-ft  runway  separation  approach  blunders  were  presented  to  all  50 
controllers  for  the  entire  25  weeks  of  testing.  Table  2-3  lists  the  approach  blunders  presented  at  4.8-s 
sensor  update  interval,  3,400-ft  runway  separation  (Weeks  1-12)  or  4,300-ft  runway  separation 
(Weeks  13-25). 
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TABLE  2-1 


Approach  Blunders  at  1.0-s  Sensor  Update  Interval 

3,400-ft  Runway  Separation  -  Weeks  1-25 
Total  s  13  Approach  Blunders 


Type: 

Single 

Fast/Slow 

Distraction 

Simult.  Missed 
Approach 

Angle: 

15 

30 

15 

• 

15 

30 

15 

• 

Range: 

(nmi) 

2-4 

8-10 

2-4 

8-10 

2-4 

8-10 

e 

B 

8-10 

fl 

8-10 

0.5 

• 

Flight  Path: 

B 

B 

B 

B 

B 

B 

B 

fl 

B 

B 

B 

fl 

e 

B 

B 

B 

B 

B 

fl 

c 

e 

Blunder  #: 

II 

B 

B 

fl 

B 

B 

B 

fl 

B 

B 

E 

B 

e 

B 

E 

B 

B 

E 

fl 

13 

e 

TABLE  2-2 


Approach  Blunders  at  2.4-s  Sensor  Update  Interval 

3,400-ft  Runway  Separation  -  Weeks  1-25 
Total  s  13  Approach  Blunders 


Type: 

Single 

Fast/Slow 

Distraction 

Simult.  Missed 
Approach 

Angle: 

(deg.) 

15 

30 

15 

* 

15 

30 

15 

e 

Range: 

(nmi) 

2-4 

8-10 

2-4 

8-10 

2-4 

8-10 

• 

fl 

8-10 

■ 

8-10 

0.5 

• 

Right  Path: 

B 

B 

B 

B 

B 

fl 

B 

II 

B 

fl 

fl 

B 

* 

B 

B 

fl 

B 

fl 

c 

• 

Blunder#: 

B 

B 

B 

B 

B 

B 

B 

B 

B 

fl 

E 

B 

• 

B 

fl 

B 

B 

E 

fl 

13 

• 

TABLE  2-3 

Approach  Blunders  at  4.8-s  Sensor  Update  Interval 

3,400-ft  Runway  Separation  •  Weeks  1-12 
4,300-ft  Runway  Separation  -  Weeks  13-25 
Total  s  1 1  Approach  Blunders 


Type: 

Single 

Fast/Sk>w 

Distraction 

Simult.  Missed 
Approach 

Angle: 

(«tog) 

15 

30 

15 

• 

15 

30 

15 

• 

Range: 

(nmi) 

2-4 

8-10 

2-4 

2-4 

8-10 

• 

H 

8-10 

B 

8-10 

g 

• 

Right  Path: 

B 

B 

B 

fl 

B 

B 

B 

fl 

B 

B 

B 

B 

• 

B 

B 

fl 

B 

B 

fl 

c 

e 

Blunder  #: 

II 

B 

fl 

fl 

fl 

fl 

B 

B 

fl 

B 

B 

B 

• 

B 

B 

fl 

B 

E 

fl 

ii 

e 

Note: 
c  -  calm 
t  -  turbulent 
*  >  not  tested 
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From  review  of  Tables  2-1  through  2-3,  it  is  seen  that  there  was  a  total  of  37  approach 
blunders  to  which  each  subject  had  the  opportunity  to  respond  during  his  or  her  week  of  testing. 

2.1.2  Research  Questions  -  Study  I 

In  studying  the  above  independent  variables  the  following  research  questions  were  asked: 

1)  How  does  sensor  update  interval  affect  reaction  time?  This  will  give  us  information 
needed  to  decide  whether  a  1.0-s  or  2.4-s  sensor  update  interval  is  required  for  system 
effectiveness  at  3,400-ft  runway  separation. 

2)  When  the  runway  separation  is  3,400  ft,  are  there  differences  in  reaction  time  attributable 
to: 

a.  the  angle  of  the  deviation  of  the  approach  blunder, 

b.  the  range  (nmi  from  the  runway  threshold)  of  aircraft  at  time  of  blunder, 

c.  the  flight  path  condition  during  the  approach  blunder, 

d.  the  type  of  approach  blunder;  for  example,  one  involving  one  aircraft  at  a  fast  speed 
deviating  toward  another  aircraft  at  a  slow  speed,  and 

e.  controller  experience  level? 

3)  Are  there  differences  in  reaction  time  attributable  to  runway  separation  in  feet  from 
centerline  to  centerline?  The  specific  case  tested  was  the  presentation  of  approach 
blunders  at  the  4.8-s  sensor  update  interval  with  3,400-ft  vs  4,300-ft  runway  separation. 

4)  When  the  runway  separation  is  4,300  ft  and  the  sensor  update  interval  is  4.8  s,  are  then 
differences  in  reaction  time  attributable  to: 

a.  deviation  angle, 

b.  range, 

c.  flight  path  condition,  and 

d.  type  of  approach  blunder? 

5)  Will  controllers  accept  the  PRM  system  as  a  means  of  safely  conducting  independent 
parallel  approaches  during  instrument  flight  rules  (IFR)  conditions  to  parallel  runways 
spaced  3,400  ft  apart? 

2.1.3  Experimental  Control  Procedures  -  Study  I 

In  order  to  obtain  reliable  and  valid  data  to  answer  the  research  questions  asked,  it  was 
necessary  to  collect  these  data  in  a  systematic  and  controlled  manner.  Therefore,  various 
experimental  controls  were  used  and  are  described  in  this  section. 

The  approach  blunders  occurred  within  the  context  of  an  "arrival  push.”  Arrival  pushes  are 
periods  of  peak  arrivals,  i.e.,  a  high  rate  of  arrivals.  The  Core  Group  of  Controllers  at  Memphis 
Airport  served  as  advisors  to  insure  the  authenticity  of  the  simulation.  According  to  the  Core  Group, 
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arrival  pushes  usually  last  for  a  maximum  of  one  hour  and  fifteen  minutes.  Therefore,  for  the 
purpose  of  realism,  the  simulated  arrival  pushes  did  not  exceed  this  time  limit 

During  the  arrival  push,  each  approach  blunder  was  presented  within  a  15-minute  period. 
Approach  blunders  occurred  at  various  times  within  the  15-minute  period  so  that  the  controller  was 
not  able  to  predict  when  they  would  occur.  The  first  approach  blunder  began  at  a  randomly  selected 
time  and  was  at  least  5  minutes  into  the  simulation.  This  was  necessary,  since  when  the  simulation 
starts,  there  are  just  a  few  aircraft  beginning  the  arrival  push  and  it  takes  about  5  minutes  for  traffic  to 
build  up  to  a  point  in  which  a  blunder  can  be  inserted  into  the  traffic. 

Figure  2-1  shows  an  example  of  the  timing  of  approach  blunders  in  a  simulated  arrival  push. 
Each  dash  (-)  represents  a  minute  in  the  15 -minute  interval  and  the  occurrence  of  an  approach 
blunder  is  represented  by  an  “x.”  A  75-minute  arrival  push  is  depicted,  i.e.,  beginning  at  0900 
hours  and  ending  at  1015  hours. 


Figure  2-1 .  Simulated  arrival  push. 

The  time  of  occurrence  for  each  approach  blunder  was  determined  through  use  of  a  random 
number  list  The  number  of  approach  blunders  per  monitoring  session  were  five  (75-minute  session), 
four  (60-minute  session),  or  three  (45-minute  session).  It  was  not  possible  to  have  the  same  number 
of  approach  blunders  in  each  arrival  push  since  arrival  pushes  needed  to  be  grouped  by  sensor 
update  interval  and  flight  path  conditions.  That  is,  all  approach  blunders  in  one  arrival  push  were  the 
same  sensor  update  interval  and  flight  path.  This  is  done  for  continuity  for  the  controllers.  It  would 
be  quite  disturbing  to  the  controller  to  keep  having  momentary  changes  in  sensor  update  interval  and 
flight  path  conditions;  and  this  would  not  represent  reality. 

Approximately  half  of  the  simulated  arrival  pushes  depicted  approaches  to  the  north  and  half 
were  to  the  south.  This  enables  du.erentiation  of  effects  that  could  be  attributed  to  direction  of 
aircraft  approach.  This  is  also  typical  of  Memphis  approaches  and  runway  assignments.  Half  of  the 
simulated  approach  blunders  occurred  on  the  right  runway  and  half  occurred  on  the  left  runway. 

Since  the  approach  blunders  were  seen  by  a  pair  of  controllers,  that  meant  that  one  controller 
in  the  pair  had  an  opportunity  to  instruct  the  endangered  aircraft,  i.e.,  giving  one  reaction  time  to  that 
approach  blunder.  In  order  to  get  the  reaction  time  of  both  controllers,  the  complete  set  of  approach 
blunders  were  presented  a  second  time.  Before  beginning  the  second  presentation,  the  controllers 
were  told  to  change  seats  so  that  the  controller  monitoring  the  right  runway  would  now  be  monitoring 
the  left  runway  and  vice  versa. 

The  second  presentation  was  different  from  the  first  in  several  ways  so  that  the  controllers 
were  not  aware  that  the  same  approach  blunders  were  being  presented  a  second  time.  In  addition  to 
the  controllers  changing  seats,  and,  therefore,  perspective,  the  flight  identification  numbers  for  aircraft 
were  changed  in  the  second  presentation  of  the  approach  blunders  so  that  the  controller  would  not 
remember  who  blundered  before  and,  therefore,  predict  an  approach  blunder.  In  their  briefing  prior 
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to  the  experiment,  they  were  told  that  some  of  the  scenarios  may  look  alike  but  were  all  unique  and, 
therefore,  must  be  attended  to  carefully.  Since  the  flight  identification  numbers  were  changed  in  the 
second  set  of  arrival  pushes,  the  audio  was  also  changed  to  correspond  to  the  new  flight 
identifications  (IDs). 

In  the  first  presentation,  there  were  nine  arrival  pushes  identified  as  Arrival  Push  1A  through 
9A.  The  second  presentation  included  the  nine  arrival  pushes  with  the  appropriate  changes,  as 
described  previously.  These  were  identified  as  Arrival  Push  IB  through  9B.  Appendix  A  lists  the 
arrival  pushes  and  approach  blunders  shown  during  the  first  twelve  weeks  of  testing.  Appendix  B  lists 
the  arrival  pushes  and  approach  blunders  shown  during  Weeks  13  through  25  of  testing. 

In  order  to  control  for  effects  which  might  be  attributed  to  order  of  presentation,  the  nine 
arrival  pushes  were  presented  in  a  counterbalanced  order.  If  all  nine  arrival  pushes  were  presented  to 
all  subjects  in  the  same  order,  for  example,  1,  2,  3,  4,  5,  6,  7,  8,  9,  then  it  could  be  said  that  subjects 
would  do  better  during  arrival  push  9  than  during  arrival  push  1.  That  is,  by  the  later  arrival  pushes, 
5  through  9,  the  subject  had  more  practice;  learning  had  occurred;  performance  had  improved. 
Therefore,  controller  pairs  were  presented  with  the  arrival  pushes  in  a  counterbalanced  order.  The 
design  has  isolated  order  as  an  experimental  variable.  If  subject  performance  improved  during  the 
later  presentations,  differences  should  average  out  across  all  subjects.  Appendix  C  shows  the 
counterbalanced  order  of  arrival  push  presentations. 

Since  half  of  the  approach  blunders  occurred  on  the  right  runway  and  half  occurred  on  the 
left  runway,  a  random  number  list  was  used  to  determine  which  approach  blunders  would  occur  on 
which  runway  and  in  which  order  of  presentation. 

Approximately  three  speed  adjustments  per  arrival  push  are  said  to  be  typical,  according  to 
information  from  the  Memphis  Core  Group  of  Controllers.  Therefore,  within  the  simulation,  three 
aircraft  per  arrival  push  flew  at  a  speed  which  we  anticipated  would  cause  the  controller  to  issue  an 
adjustment  The  number  of  aircraft  which  needed  a  speed  adjustment  was  controlled  in  an  effort  to 
keep  the  workload  at  approximately  the  same  level  for  all  controllers.  In  the  simulation,  the  Monitor 
Controllers  gave  speed  adjustments  to  our  experimental  pseudopilots,  and  the  pseudopilots 
acknowledged  the  instruction  and  made  the  adjustments.  The  role  of  the  pseudopilot  is  explained  in 
the  Section  3. 1.2.2. 1,  Experimental  Control  Room. 

2.2  STUDY  D 

After  a  preliminary  analysis  of  the  results  of  Study  I,  the  PRM  Working  Group  requested 
testing  to  look  at  the  effect  of  a  number  of  variables  during  simulation  of  monitoring  runways  with 
3,000-ft  separation.  Based  on  the  results  of  Study  I,  not  all  of  the  variables  selected  for  study  in 
Study  I  were  included  in  Study  II. 

Consideration  of  the  4.8-s  sensor  update  interval  was  excluded  from  Study  II.  The  4.8-s 
sensor  update  interval  was  considered  to  be  too  slow  an  update,  considering  the  lessened  separation 
between  runways.  It  was  determined  that  all  approach  blunders  would  occur  during  calm  rather  than 
both  calm  and  turbulent  flight  path  conditions.  It  was  also  determined  that  this  would  be  a  small 
sample  study  and,  therefore,  would  involve  the  participation  of  ten  subjects  who  were  experienced 
Monitor  Controllers.  Consequently,  this  study  did  not  include  an  examination  of  controller  reaction 
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time  differences  attributable  to  controller  experience  level,  i.e.,  novice  vs  experienced  Monitor 
Controllers. 

Changes  in  the  visual  presentations  of  the  simulations  included:  (1)  The  aiipoit  map  changed 
to  depict  runway  separation  of  3,000  ft  rather  than  the  3,400  ft  depicted  in  Study  I,  and  (2)  TNSE  was 
increased. 

TNSE  was  not  changed  as  a  result  of  Study  I  findings,  but  as  a  result  of  live  aircraft  data 
collected  at  Memphis.  Once  the  live  aircraft  approaches  recorded  at  Memphis  were  analyzed,  the 
information  was  used  to  design  an  improved  TNSE  model  for  Study  II.  For  a  detailed  analysis  of  the 
live  aircraft  data,  the  reader  is  referred  to  [2].  Figure  2-2  is  excerpted  from  that  report  The  figure  is 
based  on  the  actual  final  approach  data  collected  at  Memphis  where  dependent  approaches  were 
conducted  to  parallel  runways  separated  by  3,400  ft  Approximately  1,000  approaches  of  large  and 
heavy  aircraft  were  included  in  the  database.  The  data  were  filtered  for  stabilized  flight  starting 
before  9.4  nxni  and  were  collected  during  MVFR  (Marginal  Visual  Flight  Rules).  Based  on  these 
data,  extrapolations  were  made  to  estimate  the  percentage  of  aircraft  that  would  penetrate  the  NTZ  if 
independent  parallel  approaches  were  conducted  to  runways  separated  by  3,000  ft  and  4,300  ft 


0.0  5.0  10.0  15.0 


Range  (nmi) 

Figure  2-2.  NTZ  Penetrations  Based  on  Memphis  MVFR  Approach  Data. 

Overall  statistics  about  the  average  paths  of  aircraft  and  the  amplitudes  of  their  oscillations  were  used 
to  create  TNSE  that  was  statistically  similar  to  what  was  observed  at  Memphis. 
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The  simulated  final  approaches  were  offset  from  the  localizer  beam  by  an  angle  (measured  at 
the  runway  threshold)  selected  from  a  normal  distribution,  and  the  aircraft  oscillated  around  these 
average  flight  paths  starting  with  an  amplitude  also  selected  from  a  normal  distribution.  The  new 
TNSE  algorithm  generated  more  Caution  and  Warning  Alerts  which  were  the  result  of  the  increased 
TNSE  and  were  not  associated  with  simulated  approach  blunders. 

2.2.1  Independent  Variables  -  Study  n 

(a)  Sensor  update  interval.  Two  sensor  update  intervals,  1.0  s  and  2.4  s,  were  explored  in 
Study  II. 

(b)  Runway  separation.  All  scenarios  in  Study  II  depicted  a  runway  separation  of  3,000  ft. 

(c)  Deviation  angle.  As  in  Study  I,  the  deviation  angles  studied  were  15  deg  and  30  deg. 

(d)  Range  of  the  approach  blunder  from  runway  threshold.  As  in  Study  I,  the  approach 
blunder  ranges  staged  both  outside  and  inside  the  outer  marker  and  after  the  missed 
approach  point 

(e)  Type  of  Blunder  “Single  Type”  and  “Fast/Slow  Type”  blunders  were  studied.  As  a 
result  of  Study  I  findings,  “Distraction  Type”  and  “Simultaneous  Missed  Approach 
Blunder”  were  not  studied.  (See  Section  4.1.6  for  details.)  For  Study  II,  “Distraction 
Type”  approach  blunders  were  altered  to  be  “Single  Type”  approach  blunders  without 
distractions. 

2.2.2  Research  Questions  -  Study  II 

In  studying  the  above  independent  variables  the  following  research  questions  were  asked 
regarding  1.0-s  and  2.4-s  sensor  update  interval: 

1 .  How  does  sensor  update  interval  affect  reaction  time? 

2.  Are  there  differences  in  reaction  time  attributable  to  runway  separation,  specifically, 
3,000  ft  vs  3,400  ft? 

3.  When  the  runway  separation  is  3,000  ft  are  there  differences  in  reaction  time  attributable 
to: 

a.  the  angle  of  the  deviation  of  the  approach  blunder, 

b.  the  range  (nmi  from  the  runway  threshold)  of  aircraft  at  time  of  blunder,  and 

c.  the  type  of  blunder,  specifically,  “Single  Type”  vs  “Fast/Slow  Type?” 

4.  Will  controllers  accept  the  PRM  system  as  a  means  of  safely  conducting  independent 
parallel  approaches  during  EFR  conditions  to  parallel  runways  spaced  3,000  ft  apart? 

2.2 3  Experimental  Control  Procedures  -  Study  II 

The  same  control  procedures  that  were  used  in  Study  I  were  used  in  Study  II.  The  arrival 
pushes  that  were  used  were  a  subset  of  the  arrival  pushes  used  in  Study  I.  Appendix  D  lists  the  arrival 
pushes  and  approach  blunders  shown  during  Study  n. 
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As  in  Study  I,  the  approach  blunders  were  seen  by  a  pair  of  controllers  which  meant  that  one 
controller  in  the  pair  had  an  opportunity  to  instruct  the  endangered  aircraft,  i.e.,  giving  one  reaction 
time  to  that  approach  blunder.  In  order  to  determine  the  reaction  time  of  both  controllers,  the 
complete  set  of  approach  blunders  were  presented  a  second  time.  Before  beginning  the  second 
presentation,  the  controllers  were  told  to  change  seats  so  that  the  controller  monitoring  the  right 
runway  would  now  be  monitoring  the  left  runway  and  vice  versa.  Therefore,  controllers  viewed  an  A 
and  B  version  of  each  arrival  push,  as  described  in  Section  2.1.3,  Experimental  Control  Procedures  - 
Study  I.  In  order  to  control  for  effects  which  might  be  attributed  to  order  of  presentation,  the  nine 
arrival  pushes  were  presented  in  a  counterbalanced  order.  As  in  Study  I,  half  of  the  approach 
blunders  occurred  on  the  right  runway  and  half  occurred  on  the  left  runway.  Approximately  half  of 
the  simulated  arrival  pushes  depicted  approaches  to  the  north  and  half  were  to  the  south.  A  random 
number  list  was  used  to  determine  which  approach  blunders  would  occur  on  which  runway  and  in 
which  order  of  presentation.  Finally,  like  Study  I,  the  opportunity  for  approximately  three  speed 
changes  was  included  in  each  arrival  push. 
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3.  METHODS 


In  Sections  3.1  and  3.2,  respectively,  the  methods  for  Study  I  and  Study  n  are  discussed.  The 
methods  for  both  studies  were  similar,  i.e.,  the  same  experimental  set-up,  including  procedures  and 
equipment,  was  used  in  both  studies.  The  studies  differed  in  the  runway  separation  used  in  the 
simulation,  the  number  of  controllers  involved,  and  the  number  and  type  of  arrival  pushes  presented. 
In  Section  3.3,  the  data  collection  and  analysis  performed  in  both  studies  is  discussed. 

3.1  METHODS  USED  IN  STUDY  I 

3.1.1  Subjects 

Twenty-five  pairs  of  controllers,  i.e.,  fifty  controllers,  participated  as  subjects  in  Study  I. 
Each  pair  of  controllers  participated  for  five  days.  Twelve  pairs  of  controllers  were  horn  Memphis 
Airport,  and  thirteen  pairs  of  controllers  were  guest  controllers,  i.e,  from  airports  where  IFR 
(instrument  flight  regulations)  simultaneous  parallel  approaches  are  conducted.  The  guest  controllers 
had  experience  in  the  Monitor  Controller  positioa  The  Memphis  controllers  did  not  have  experience 
in  the  Monitor  Controller  position.  Therefore,  there  were  novice  and  experienced  Monitor 
Controllers.  This  was  done  in  order  to  enable  a  comparison  of  the  effectiveness  of  the  PRM  system 
relative  to  experience  level  of  controller.  The  experienced  Monitor  Controllers  were  from  the 
following  airports: 

Atlanta,  GA  Houston,  TX 

Charlotte,  SC  Los  Angeles,  CA 

Dallas/Fort  Worth,  TX  Pittsburgh,  PA 

Denver,  CO 

3.1.2  Facilities  and  Equipment 

3. 1 .2. 1  The  PRM  Display 

The  display  consists  of  a  large  (20"  x  20"),  high-resolution,  color  monitor.  When  the  displays 
are  implemented.  Monitor  Controllers  will  use  the  PRM  displays  in  the  Terminal  Radar  Approach 
Control  (TRACON)  IFR  room.  Associated  with  the  displays  is  the  same  communications  equipment 
and  ancillary  data  displays  found  in  the  plan  view  displays  (PVDs)  in  use  today. 

In  addition  to  color,  resolution,  and  size,  the  PRM  display  differs  from  the  traditional  PVD  in 
other  important  ways.  One  feature  is  the  ability  to  display  a  “predictor”  that  projects  where  the 
aircraft  will  be  in  a  specified  period  of  time,  if  the  aircraft  were  to  continue  on  its  present  course.  The 
Monitor  Controller  can  select  a  predictor  of  0,  2,  4,  6,  8,  or  10  s.  (For  these  studies,  the  predictor  was 
set  for  prediction  of  position  in  the  next  10  s.)  Another  feature  is  the  ability  to  expand  the  axis 
perpendicular  to  the  runways  compared  with  the  axis  along  the  runways,  which  has  the  effect  of 
making  lateral  deviations  more  evident  to  the  controller.  The  display’s  expansion  capability  is  up  to 
32  times  the  actual  dimensions.  A  4:1  expansion  was  used  during  the  studies. 

Perhaps  the  most  significant  feature  of  the  PRM  display  is  the  automated  alerts.  The  system 
provides  two  alerts  to  the  Monitor  Controller  on  the  occurrence  of  possible  hazardous  flight  path 
deviations.  When  it  predicts  that  the  aircraft  will  enter  the  NTZ  (no  transgression  zone)  within  the 
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specified  time,  i.e.,  10  s,  the  aircraft  symbol  and  accompanying  data  tag  of  the  blundering  aircraft 
changes  from  green  to  yellow,  and  an  audible  alert  is  delivered.  The  audible  alert  is  provided  by  a 
synthesized  voice  (DECtalk  DTC01),  proclaiming  “Caution!”  and  the  flight  ID  of  the  aircraft  that  is 
deviating  toward  the  zone.  When  the  aircraft  has  penetrated  the  NTZ,  the  aircraft  symbol  and 
accompanying  data  tag  of  the  blundering  aircraft,  as  well  as  the  aircraft  symbol  and  accompanying 
data  tag  of  the  endangered  aircraft,  change  color  to  red  and  the  synthesized  voice  proclaims 
“Warning!”  The  Core  Group  of  Controllers  made  the  decision  that  the  flight  ID  of  the  endangered 
aircraft  was  not  necessary  at  this  time  since  it  had  been  previously  given. 

Because  Monitor  Controllers  may  not  see  a  blunder  for  many  months  due  to  the  relative 
rarity  of  approach  blunders,  the  alerts  are  valuable.  An  alert  will  confirm  that  an  approach  blunder  is 
occurring,  allowing  the  controller  to  detect  the  approach  blunder  earlier  and  take  prompt  corrective 
action. 

The  alert  system  design  provides  two  levels  of  alerts:  (1)  one  alert  which  is  very  sensitive  to  a 
blunder  and  gives  an  early  warning,  sometimes  unnecessarily,  and  (2)  a  second  alert  which  is  less 
sensitive  and  does  not  occur  until  actual  penetration  of  the  zone  has  occurred.  As  a  safeguard, 
redundancy  is  built  into  these  two  alerts.  As  previously  mentioned,  the  alerts  are  both  visual  and 
audible.  Therefore,  if  a  controller  is  distracted  from  looking  at  the  screen,  and  an  approach  blunder 
occurs,  the  alert  may  not  have  been  seen,  but  it  may  be  heard.  Table  3-1  summarizes  the  types  of 
alerts,  their  meaning,  and  their  form  of  visual  and  audible  presentation. 

Table  3-1 

PRM  Alerting  System 


Type  of  Alert 

Meaning 

Visual  Alert 

Audible  Alert 

Caution  Alert 

NTZ  penetration  is  pre¬ 
dicted  to  occur  in  10  s  or 
less 

color  change  from 
green  to  yellow 

'Caution!  “(Flight  ID  is 
given)" 

Warning  Alert 

NTZ  penetration  has 
occurred 

color  change  to  red 

“Warning!" 

3. 1.2.2  The  Testing  Environment 

Study  I  was  conducted  at  the  PRM  Site  at  Memphis  International  Airport  from  February  5  to 
August  3,  1990.  A  photograph  of  the  PRM  Site  is  shown  in  Figure  3-1.  This  is  a  facility  which  was 
constructed  for  the  sole  purpose  of  Mode  S  data  collection.  For  the  experiment,  two  rooms  at  the  site 
were  used.  Figure  3-2  depicts  the  configuration  of  the  Controller  Response  Test  Facility. 

3.1. 2.2.1  Experiment  Control  Room 

The  Experiment  Control  Room  is  depicted  on  the  left  side  of  Figure  3-2.  In  the  Experiment 
Control  Room  were  two  pseudopilots,  the  Simulation  Coordinator,  and  all  the  equipment  needed  to 
direct  the  pre-recorded  simulation  to  the  controllers  and  to  record  all  data.  There  were  two  DEC 
pVAX  computers  executing  the  PRM  software  communicating  over  serial  lines.  The  upstream  VAX 
executed  the  simulation  and  passed  data  on  to  the  downstream  VAX.  Each  VAX  sent  graphics 
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Figure  3-1.  MIT  Lincoln  Laboratory  PRM  site  at  the  Memphis  International  Airport. 
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commands  to  its  own  Metheus  Graphics  Processor  which  drove  one  of  the  Sony  displays  viewed  by 
the  controllers  in  the  Subject  Participation  Room.  One  VAX  also  drove  the  DECtalk  speech 
synthesizer  that  spoke  alerts  (described  in  Section  3. 1.2.1)  when  aircraft  neared  or  entered  the  NTZ. 

In  the  Experiment  Control  Room  the  simulation  appeared  on  two  19-inch  VAX  system 
monitors.  The  Simulation  Coordinator  controlled  the  presentation  of  the  simulation  through  use  of 
two  monitors  and  accompanying  keyboards.  The  simulation  was  observed  by  the  pseudopilots  on  the 
same  two  monitors  used  by  the  Simulation  Coordinator.  This  enabled  them  to  see  what  the 
controllers  were  seeing.  Each  pseudopilot  was  responsible  for  one  runway.  He  watched  the  screen 
and  followed  scripts  which  specified  the  pre-programmed  occurrence  of  each  approach  blunder  and 
potential  speed  change  which  the  controller  monitoring  that  runway  might  make. 

When  instructed  by  the  controller,  the  respective  pseudopilot  entered  data  through  use  of  the 
graphic  input  device  on  his  own  Macintosh  PC,  which  displayed  a  left-turn  compass  rose,  a  right-turn 
compass  rose,  a  speed  scale,  an  altitude  scale,  and  a  list  of  the  currently  active  aircraft  flight  IDs. 
Figure  3-3  depicts  the  features  of  the  Pseudopilot  Interface  Screen.  He  would  set  up  a  left  or  right 
turn  to  a  new  heading,  a  new  altitude,  and  a  new  speed. 

The  pseudopilot  knew  (from  the  script)  when  the  next  scheduled  approach  blunder  would 
occur.  Before  it  occurred,  he  used  a  mouse  to  set  up  the  flight  parameters  that  were  expected  to  be 
given  in  the  breakout  instruction  from  the  controller.  When  the  command  to  break  out  came,  the 
pseudopilot  clicked  on  a  send  command  and  the  new  flight  parameters  were  sent  to  the  simulation 
computer  after  a  period  of  time  randomly  selected  by  the  software  to  represent  delay  in  the  cockpit 
In  the  case  of  speed  changes  only,  the  new  speed  was  sent  without  delay.  Sometimes  the  controllers 
would  order  heading  changes  to  return  to  the  localizer.  These  instructions  would  be  to  aircraft 
depicting  flight  paths  affected  by  simulated  turbulent  conditions  or  to  aircraft  demonstrating  flight 
path  deviations  in  the  distraction  blunder  scenarios.  The  pseudopilot  would  acknowledge  the 
instruction  but  would  take  no  action,  knowing  the  simulation  was  depicting  a  momentary  deviation 
and  that  the  aircraft  would  return  to  course  without  pseudopilot  intervention. 

The  Audio  Tape  Playback  was  synchronized  with  the  visual  presentation  of  each  Arrival  Push. 
This  was  a  pre-recorded  background  audio  which  the  controller  heard  during  each  Arrival  Push.  It 
contained  the  standard  radio  transmissions  that  are  made  by  local  controllers  and  pilots  on  final 
approach.  (The  voice  of  one  pilot  was  pre-recorded  to  simulate  all  pilot  communications.) 

The  controller  monitoring  the  right  runway  heard  the  audio  of  the  right  local  controller  and 
the  pilot  assigned  to  land  on  the  right  runway.  The  controller  monitoring  the  left  runway  heard  the 
audio  of  the  left  local  controller  and  the  pilot  assigned  to  land  on  the  left  runway. 

The  Audio  Tape  Recorder  was  used  to  record  all  radio  transmissions  made  by  the  subjects. 
This  recording  was  also  synchronized  with  the  visual  presentation  of  the  Arrival  Pushes.  This 
recording  was  used  to  obtain  the  controller  reaction  time  for  each  approach  blunder. 
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3. 1.2.2. 2  Subject  Participation  Room 

In  an  operational  PRM,  the  PRM  displays  would  be  in  the  TRACON  control  room,  near  the 
other  radar  controllers  managing  traffic  in  the  terminal  area.  During  the  simulation,  the  controllers 
were  seated  in  a  room  containing  only  the  equipment  necessary  for  parallel  approach  monitoring. 
The  more  sterile  environment,  though  lacking  the  activity  and  thus  some  of  the  realism  of  a  control 
room,  ensured  that  the  simulation  would  not  interfere  with  ongoing  TRACON  operations.  It  also 
allowed  better  control  of  the  experiment. 

The  Subject  Participation  Room  is  depicted  on  the  right  side  of  Figure  3-2.  An  Approach 
Control  Console  was  provided  for  each  controller.  The  Console  included  the  20-inch  SONY  display. 
As  indicated  by  the  arrows  shown  in  Figure  3-2,  each  controller  was  provided  with  a  communication 
system  which  was  connected  to  the  Audio  Tape  Playback  unit,  the  Audio  Tape  Record  unit,  and  the 
appropriate  pseudopilot. 

Figure  3-4  is  a  photograph  of  The  Subject  Participation  Room.  During  testing  the  lighting 
was  controlled  to  simulate  the  dark  environment  of  an  actual  TRACON. 

The  Test  Conductor  was  with  the  controllers  in  the  Subject  Participation  Room.  The  Test 
Conductor’s  responsibility  was  to  ensure  that  all  procedures  were  properly  followed  and  to  record  any 
significant  interactions  between  the  controllers  and  anomalies  in  environment  or  Arrival  Push 
presentations  which  may  have  affected  the  interpretation  of  the  data  collected.  The  Test  Conductor 
was  able  to  switch  between  hearing  the  controllers,  the  pseudopilots,  and  the  Audio  Tape  Playback. 

3. 1.2.3  The  Simulation 

The  simulation  was  an  audio-visual  presentation  of  approach  blunders  occurring  during 
eighteen  Arrival  Pushes  (see  Appendix  A  and  B  for  a  list  of  approach  blunders  in  each  Arrival  Push). 
The  audio  and  visual  portions  of  the  simulation  were  made  as  realistic  as  possible  within  the 
constraints  of  the  experiment  The  limitations  inherent  in  a  simulation  are  recognized.  Simulations 
cannot  completely  replicate  the  conditions  experienced  in  “real  life,”  i.e.,  actual  air  traffic  control 
operations. 

3. 1.2. 3.1  The  Visual  Portion  of  the  Simulation 

The  density  of  traffic,  type  of  aircraft  represented,  altitudes,  speeds,  and  headings,  were  all 
based  on  actual  traffic  information  from  the  Memphis  tower  staff  and  tapes  of  actual  traffic  at 
Memphis.  Numerous  contacts  were  made  with  Memphis  Air  Traffic  Control  personnel  to  verify  and 
supplement  these  data. 

The  typical  flight  path  for  all  but  small  General  Aviation  (GA)  aircraft  was  to  enter  the 
display  area  on  the  base  leg  at  190  knots  and  slow  to  170  knots  on  final  at  an  altitude  of  3,000  ft  or 
4,000  ft,  depending  on  the  runway.  Small  aircraft  flew  final  at  80  or  90  knots.  At  a  point  9  or 
10  nmi  out,  the  altitudes  for  both  approach  paths  became  the  same  and  the  aircraft  followed  a  3-deg 
glide  slope,  slowing  to  140  knots  over  the  outer  marker  and  on  down  to  120  knots  at  the  threshold. 
Spacing  was  about  3  nmi  along  the  flight  path  with  aircraft  flying  simultaneous  approaches  except 
when  unusual  spacing  requirements  temporarily  precluded  it  Heavy  aircraft  were  kept  5  nmi  in  front 
of  any  following  aircraft.  All  turns  were  made  at  a  standard  n»te  of  3  deg  per  second,  and 
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Figure  3-4.  The  Subject  Participation  Room.  215045-1 


normal  accelerations  were  at  ±2.5  knots  per  second.  Takeoff  accelerations  were  at  5  knots  per 
second.  A  sinusoidal  Flight  Technical  Error,  or  wobble,  was  added  to  the  aircraft's  final  approach 
path  with  a  period  of  about  two  minutes  and  an  amplitude  at  a  range  of  15  nmi  of  about  500  ft  in 
“turbulent"  conditions  or  150  ft  in  “calm”  conditions.  The  amplitude  of  the  wobble  decreased 
linearly  to  zero  at  touchdown.  Appendix  E  shows  the  airline  and  aircraft  type  mix. 

3.1. 2.3.2  The  Audio  Portion  of  the  Simulation 

These  recordings  were  made  by  an  experienced  pilot  and  a  retired  air  traffic  controller,  in 
order  to  enhance  the  authenticity  of  the  simulation.  Using  pre-recorded  audio  also  enabled 
consistency  throughout  the  study,  i.e.,  all  controllers  heard  the  same  audio  each  week. 

3. 1.2.4  Features  of  the  PRM  Display 

The  geographic  maps  and  aircraft  positions  were  displayed  on  a  Sony  DDM-2801C  2K  x  2K 
color  monitor  driven  by  a  Metheus  £23720  graphics  controller.  The  maps  contained  most  of  the 
features  on  the  Automated  Radar  Terminal  System  (ARTS)  video  maps  currently  used  by  the 
Memphis  TRACON.  Of  these,  the  less  important  map  features  include  secondary  airports,  major 
geographical  features  (e.g.,  rivers  and  coastlines),  obstacles,  navigational  reference  lines,  and 
navigational  checkpoints.  The  more  important  features  (refer  to  Figure  3-5)  are  the  parallel  runways 
and  their  extended  centerlines;  three  parallel  reference  lines,  200  ft  apart,  running  along  the  inside  of 
each  extended  center  line;  and  the  NTZ  covering  a  width  of  2,000  ft  centered  between  the  extended 
centerlines.  Runways  not  monitored  were  shown  for  reference.  The  extended  centerlines  are  broken 
into  1-mile  lengths  separated  by  1-mile  spaces  and  range  out  to  15  miles  from  the  runway  threshold 
on  the  approach  side  and  to  5  miles  on  the  departure  side. 

An  aircraft  position  is  displayed  as  a  small  circle  connected  by  a  leader  line  to  a  three-line 
data  block.  The  information  displayed  in  the  data  block  lines  consists  of  (1)  the  flight  ID,  (2)  the 
altitude  and  ground  speed,  and  (3)  the  assigned  runway  code  and  aircraft  type.  There  can  be  a 
projection  line  showing  the  predicted  position  up  to  10  s  into  the  future  and  a  history  trail  of  up  to 
six  past  positions.  This  ensemble  is  normally  green  except  for  the  predictor  line,  which  is  cyan.  But 
when  an  aircraft  is  predicted  to  be  inside  the  active  NTZ  within  10  s,  it  turns  yellow  and  a  spoken  alert 
sounds  (e.g.,  “Caution!  Northwest  twelve-sixty-four”).  If  the  aircraft  actually  enters  the  NTZ,  the 
aircraft  ensemble  turns  red  and  a  different  spoken  alert  sounds,  i.e.,  “Warning!”  The  symbology 
and  identifier  for  the  aircraft  will  also  turn  red  if  the  tracking  algorithm  determines  that  it  is  in  danger 
from  another  aircraft,  or  that  it  is  approaching  the  wrong  runway,  or  that  its  transponder  has  failed. 

Figure  3-5  shows  the  PRM  display  in  negative  as  seen  on  the  Sony  monitors.  With  runways 
oriented  vertically  on  the  screen,  not  all  the  screen  area  in  the  x-axis  is  needed  to  display  aircraft. 
Therefore,  five  auxiliary  windows  are  placed  permanently  on  the  right  hand  margin.  From  top  to 
bottom,  the  windows  are  (1)  the  time,  barometer,  and  stopwatch  window,  (2)  the  status  window 
showing  current  zoom  factors,  predictor  time,  number  of  history  positions,  and  the  altitude  range 
being  displayed,  as  well  as  instructions  for  and  status  of  menu  selections  in  progress,  (3)  the  sub¬ 
menu  window  required  by  certain  main  menu  selections  for  entering  data,  (4)  the  main  “point-and- 
click”  menu,  and  (5)  the  secondary  graphics  window  that  shows  the  area  within  a  blue  box  that 
can  be  moved  around  the  primary  graphics  window.  (The  contents  of  this  window  are  always 
displayed  undistorted.) 
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3. 1.2. 5  The  PRM  Display  Testing  Configuration 

The  PRM  Display  Testing  Configuration  was  determined  by  the  PRM  Working  Group. 
During  the  two  studies,  the  controllers  only  saw  the  primary  graphics  windows.  Since  the  menus  are 
only  used  to  set  up  the  display  and  do  not  need  to  be  present  during  approach  monitoring,  they  were 
omitted.  The  primary  graphics  window  can  be  zoomed  independently  in  the  x  and  the  y  axes.  There 
are  two  benefits  that  result  from  stretching  the  cross-runway  dimension  more  than  the  other.  First, 
cross-runway  resolution  is  increased  so  that  the  200-ft  parallel  reference  lines  can  be  seen  without 
losing  the  ends  of  the  approach  path  off  the  top  and  bottom  of  the  screen.  Second,  cross-runway 
motion  is  exaggerated  which  causes  the  observer  to  be  more  sensitive  to  motion  toward  the  NTZ.  The 
ratio  of  x-  and  y-axis  zoom  factors  used  during  the  two  studies  was  4:1.  During  the  two  studies,  the 
predictor  line  was  displayed  but  not  the  history  trail. 

3.1.3  Study  Procedures 

3. 1.3.1.  Testing  Schedule 

For  each  pair  of  controllers,  Monday  of  each  week  was  devoted  to  familiarization  with  the 
PRM  program,  operations  of  the  monitor  display,  and  independent  parallel  approach  procedures. 
During  the  week,  testing  was  conducted  within  eighteen  arrival  pushes.  Each  arrival  push  was  a 
simulation  of  approximately  one  hour  of  approach  traffic,  during  which  approach  blunders  occurred. 
Debriefing  and  completion  of  the  Controller  Opinion  Survey  occurred  on  Friday. 

Table  3-2  shows  the  Subject  Schedule  followed  for  Pair  #1.  This  particular  schedule  is  one 
example  of  the  general  schedule  adhered  to  by  each  pair  of  controllers.  All  controllers  had  a  similar 
schedule  over  the  five-day  period.  However,  since  arrival  pushes  were  presented  in  a  counterbalanced 
order,  the  schedule  varied  in  accordance  with  the  counterbalanced  order  of  presentations  of  arrival 
pushes.  For  example.  Pair  #1  began  testing  with  Arrival  Push  1A,  Pair  #2  began  testing  with  Arrival 
Push  2A,  Pair  #3  began  testing  with  Arrival  Push  3A,  and  so  forth  through  Pair  #25. 
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TABLE  3-2 

Subject  Schedule  for  Study  I 
(One  Pair  of  Subjects  Per  Week) 

This  sample  schedule  was  followed  for  Pair  #1. 


Day  1 

Day  4 

En^SMMH 

Program  Orientation 

1  hr 

Arrival  Push  1A 

75  min 

5  blunders 

Arrival  Push  6A 

60  min 

4  Blunders 

Arrival  Push  2B 

60  min 

4  Blunders 

Arrival  Push  7B 

60  min 

4  Blunders 

BREAK 

BREAK 

BREAK 

BREAK 

BREAK 

Independent  Parallel 
ILS  Approach 
Procedures 

2  hr 

Arrival  Push  2A 

60  min 

4  Blunders 

Arrival  Push  7A 

60  min 

4  Blunders 

Arrival  Push  3B 

60  min 

4  Blunders 

Arrival  Push  8B 

60  min 

4  Blunders 

BREAK 

BREAK 

BREAK 

BREAK 

BREAK 

Monitor  Display 
Operations  Briefing 
&  Lab 

1  to  2  hr 

Arrival  Push  3A 

60  min 

4  Blunders 

Arrival  Push  8A 

60  min 

4  Blunders 

Arrival  Push  4B 

75  min 

5  Blunders 

Arrival  Push  9B 

45  min 

3  Blunders 

BREAK 

BREAK 

BREAK 

BREAK 

BREAK 

Hands-on  Practice  & 
Dry-Run 

2  hr 

Arrival  Push  4A 

75  min 

5  Blunders 

Arrival  Push  9A 

45  min 

3  Blunders 

Arrival  Push  5B 

60  min 

4  Blunders 

Debriefing 

1  1/2  hr 

BREAK 

BREAK 

BREAK 

Arrival  Push  5A 

60  min 

4  Blunders 

Arrival  Push  IB 

75  min 

5  Blunders 

Arrival  Push  6B 

60  min 

4  Blunders 

3. 1.3. 2  Subject  Training 

For  each  pair  of  controllers,  Day  1  of  the  study  was  dedicated  to  training.  The  training 
included: 

1.  Program  Orientation.  Controllers  were  given  background  and  goals  of  the  PRM 
Program;  a  general  description  of  the  system  and  the  display;  discussion  of  the 
responsibilities  of  the  Monitor  Controller,  and  discussion  of  their  role  as  subjects. 

2.  Independent  Parallel  Approach  Procedures.  Controllers  were  briefed  on  local  area 
familiarization,  approach  charts  and  procedures,  obstructions,  localizer  turn-on  and 
Monitor  Controller  responsibility,  definition  of  the  alerts  in  the  system,  missed- 
approach  procedure,  and  communications. 

3.  Monitor  Display  Operation.  Controllers  were  given  a  briefing  on  display  capabilities, 
menus,  defaults,  keyboard,  and  automatic  alerts. 

4.  Dry-Run/Practical  Demonstration.  Controllers  saw  and  heard  a  sample  arrival  push 
simulation  and  had  the  opportunity  to  react  to  the  simulated  approach  blunders.  This 
was  for  training  purposes  and  the  reaction  time  data  were  not  included  in  the  data 
analysis. 
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3. 1.3. 3  Monitoring  Session 

One  pair  of  controllers  participated  each  week  in  the  Monitoring  Sessions.  The  controllers  sat 
next  to  each  other,  each  observing  the  respective  runway  to  which  he/she  had  been  assigned.  One 
controller  monitored  the  right  parallel  runway,  and  one  monitored  the  left  parallel  runway.  Each 
controller  had  a  PRM  Monitor  Console,  which  housed  the  SONY  display  and  keyboard.  Each 
controller  had  his/her  own  headset  on  which  to  communicate.  Each  controller  heard  the  audio  of  the 
respective  local  controller  and  pilots  for  the  assigned  runway.  The  information  seen  and  heard  by  all 
pairs  was  the  same. 

Before  each  arrival  push  began,  the  controllers  were  briefed  in  accordance  with  the  routine 
briefing  experienced  by  a  Monitor  Controller.  The  controllers  were  given  weather  information  and 
were  told  the  sensor  update  interval  at  which  information  would  be  provided. 

Controllers  were  instructed  to  view  and  listen  to  each  arrival  push  and  to  perform  the  duties  of 
a  Monitor  Controller.  The  instructions  regarding  breaking-out  an  endangered  aircraft  were: 

“When,  in  your  best  judgment,  penetration  of  the  NTZ  is  imminent,  instruct  aircraft 

on  the  adjacent  final  approach  course  to  alter  course  to  avoid  the  deviating  aircraft.” 

The  controller  was  instructed  that  he/she  did  not  have  to  wait  until  actual  penetration  occurred  (as 
signaled  by  the  red  alert  warning  and  DECtalk).  Through  each  controller's  headset,  the  PRM  voice 
alerts  were  heard.  If  the  controller  believed  that,  in  his/her  best  judgment,  penetration  was  imminent, 
he/she  should  respond  immediately.  The  instructions  to  be  given  to  the  pilot  of  the  endangered 
aircraft  were:  “TURN  (lefl/right)  IMMEDIATELY,  HEADING  (degrees),  CLIMB  AND  MAINTAIN 
(altitude).” 

The  audio  was  set  up  so  that  the  controller  could  override  the  local  controller  and  speak 
directly  to  a  pilot  This  gave  the  controller  quick  access  for  giving  a  breakout  instruction.  The 
controller  could  also  override  the  local  controller  and  give  speed  adjustments  if  necessary.  A 
pseudopilot  acknowledged  all  communications  from  the  controller. 

After  each  arrival  push  was  completed,  the  controllers  completed  a  short  debriefing 
questionnaire.  This  questionnaire  was  designed  to  elicit  controller  opinion  on  the  realism  of  the 
simulation.  After  a  15-minute  break,  the  next  routine  briefing  occurred,  followed  by  presentation  of 
the  next  arrival. 


3. 1.3.4  Debriefing 

A  Controller  Opinion  Survey  was  used  to  obtain  the  opinions  of  the  controllers  on  the 
effectiveness  of  the  PRM  system  and  its  overall  acceptability  for  use.  On  the  first  day  of  participation 
in  the  study,  the  pair  of  controllers  were  given  a  copy  of  the  survey  which  was  to  be  completed  at  the 
end  of  the  week.  This  was  done  so  that  throughout  the  week  the  controllers  could  be  mindful  of  the 
various  areas  in  which  their  opinions  were  needed.  On  Friday,  after  all  testing  was  completed,  each 
controller  filled  out  the  survey.  A  copy  of  the  Survey  is  found  in  Appendix  F. 
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3.2  METHODS  USED  IN  STUDY  H 


3.2.1  Subjects 

Five  pairs  of  controllers,  i.e.,  ten  controllers,  participated  as  subjects  in  Study  II.  All  ten 
controllers  had  participated  in  Study  I  and  volunteered  to  participate  in  Study  II.  All  were 
experienced  Monitor  Controllers  from  airports  where  IFR  simultaneous  parallel  approaches  are 
conducted.  The  controllers  were  from  the  following  airports: 

Los  Angeles,  CA  Pittsburgh,  PA 

Atlanta,  GA  Houston,  TX 

Each  pair  of  controllers  participated  for  two  and  a  half  days. 

3.2.2  Facilities  and  Equipment 

The  same  facilities  and  equipment  were  used  in  both  Study  I  and  Study  n.  See  Section  3.1.2 
for  details. 

3.2.2. 1  The  Testing  Environment 

The  study  was  conducted  at  the  PRM  Site  at  Memphis  International  Airport  from 
November  26  through  December  12,  1990.  The  PRM  Controller  Response  Test  Facility  was  set  up  in 
the  same  manner  for  both  Study  I  and  Study  II.  See  Test  Environment  previously  discussed  in 
Section  3. 1.2.2. 

3.2.2. 2  The  Simulation 

In  Study  I,  eighteen  arrival  pushes  were  presented  to  the  controllers.  In  Study  II,  eight  arrival 
pushes  selected  from  those  used  in  Study  I,  were  presented  with  the  new  map  showing  a  3,000-ft 
runway  separation.  Ten  of  the  controllers  who  participated  in  Study  I  also  participated  in  Study  II. 
Section  2.2  explains  why  certain  arrival  pushes  were  eliminated  in  Study  II.  Appendix  D  includes  the 
list  of  arrival  pushes  used  in  Study  II.  The  same  audio  presentations  which  accompanied  each  visual 
presentation  were  used  in  both  studies. 

3.2.3  Study  Procedures 

3.2.3. 1  Testing  Schedule 

A  sample  Subject  Schedule  is  presented  below.  This  particular  schedule  is  one  example  of 
the  general  schedule  adhered  to  by  each  pair  of  controllers.  All  controllers  had  a  similar  schedule 
over  the  three-day  period.  However,  since  arrival  pushes  were  presented  in  a  counterbalanced  order, 
the  schedule  varied  in  accordance  with  the  counterbalanced  order  of  presentations  of  arrival  pushes. 
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TABLE  3-3 


Subject  Schedule  for  Study  II 
(Two  Pairs  of  Subjects  Per  Week) 

This  sample  schedule  was  followed  for  Pair  #1  and  Pair  #2. 


Day  1 

Day  2 

USMMKKM 

Day  4 

EEHESHiHi 

PAIR  #1 

Refresher  Course 
and  Training 

Arrival  Push 

3  hr 

Arrival  Push  4A 

75  min 

5  Blunders 

Arrival  Push  5B 

60  min 

4  Blunders 

Arrival  Push  2A 

60  min 

4  Blunders 

Arrival  Push  4B 

75  min 

5  Blunders 

BREAK 

BREAK 

BREAK 

BREAK 

Arrival  Push  5A 

60  min 

4  Blunders 

Debriefing 

1  1/2  hr 

Arrival  Push  4A 

75  min 

5  Blunders 

Arrival  Push  5B 

60  min 

4  Blunders 

BREAK 

PAIR  *2 

BREAK 

BREAK 

Arrival  Push  1 B 

75  min 

5  Blunders 

Refresher  Course 
and  Training 

Arrival  Push 

3  hr 

Arrival  Push  5A 

60  min 

4  Blunders 

Arrival  Push  IB 

75  min 

5  Blunders 

BREAK 

Arrival  Push  1A 

75  min 

5  Blunders 

BREAK 

BREAK 

BREAK 

Arrival  Push  2B 

60  min 

4  Blunders 

Arrival  Push  1A 

75  min 

5  Blunders 

Debriefing 

BREAK 

Arrival  Push  2A 

60  min 

4  Blunders 

Arrival  Push  4B 

75  min 

5  Blunders 

Arrival  Push  2B 

60  min 

4  Blunders 

3.2. 3. 2  Subject  Training 

Since  all  ten  controllers  participated  in  Study  I,  extensive  training  in  the  PRM  system  was  not 
necessary.  A  brief  refresher  course  was  given  for  Study  II,  and  then  controllers  participated  in  a 
Training  Arrival  Push.  Approach  blunders  occurred,  controllers  responded,  but  data  were  not 
included  in  the  data  analysis.  This  practice  session  provided  an  opportunity  for  the  controllers  to  ask 
any  questions  about  the  simulation  or  testing  procedures. 

3.2. 3. 3  Monitoring  Sessions 

The  same  basic  instructions  were  used  in  both  studies.  However,  two  additions  were  made  to 
the  instructions  for  Study  II: 

(1)  Controllers  were  reminded  that  when  they  participated  in  Study  I,  the  runway  separation 
was  3,400  or  4,300  ft,  and  in  this  simulation,  the  runway  separation  is  3,000  ft. 

(2)  Controllers  were  told  that  the  runways  are  closer  together  in  this  simulation,  and  the 
NTZ  is  still  2,000  ft.  Therefore,  the  Normal  Operating  Zone  (NOZ)  is  decreased.  This 
means  that  it  is  more  difficult  for  pilots  to  stay  within  the  NOZ,  and  there  is  a  greater 
likelihood  that  yellow/Caution  Alerts  (predicting  NTZ  penetration)  and  red/Waming 
Alerts  (indicating  NTZ  penetration)  will  occur. 
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3.2. 3.4  Debriefing 

The  Controller  Opinion  Surveys  used  in  Studies  I  and  II  were  similar  with  the  following 
exceptions.  Since  in  Study  I  the  controllers  had  already  been  asked  about  their  opinions  on  the 
Monitor  Controller  Functions,  NTZ  Alerts,  and  Display  Information  Content  and  Presentation,  there 
was  no  need  to  repeat  these  questions  in  Study  II.  Questions  in  Study  II  included  the  General 
Acceptance  section,  asking  controllers  their  opinion  of  the  acceptability  of  the  system  when  runway 
separation  is  3,000  ft  Questions  in  Study  II  also  included  questions  regarding  the  realism  of  the 
simulation.  A  copy  of  the  Survey  is  included  in  Appendix  G. 

3.3  DATA  COLLECTION  AND  ANALYSIS  IN  STUDY  I  AND  STUDY  H 

In  both  Study  I  and  Study  II,  three  types  of  data  were  collected  in  order  to  evaluate  the 
effectiveness  of  the  system.  Sections  3.3.1  through  3.3.3  define  each  type  of  data  and  explain  how 
the  data  were  collected. 

3.3.1  Controller  Blunder  Resolution 

Before  acting  to  resolve  an  approach  blunder,  the  controller  pair  monitoring  the  two 
approaches  must  first  decide  that  a  blunder  is  occurring.  The  Monitor  Controller  observing  the 
blundering  aircraft  will  instruct  that  aircraft  to  return  to  course.  If  that  aircraft  does  not  return  to 
course  or  if  safety  does  not  permit  waiting  to  see  if  that  aircraft  returns  to  course,  the  Monitor 
Controller  observing  the  endangered  aircraft  must  transmit  a  breakout  instruction  to  the  endangered 
aircraft.  The  sooner  the  instruction  is  transmitted  to  the  endangered  aircraft,  the  more  time  is 
available  to  maneuver  the  endangered  aircraft  out  of  the  way.  Thus,  the  response  time  of  the 
controller  monitoring  the  endangered  aircraft  is  a  critical  element  in  a  successful  blunder  resolution. 
Procedure  sets  limits  on  the  controller  response.  Safe  separation  is  lost  whenever  an  aircraft  enters  the 
NTZ,  and  the  controller  must  take  action,  usually  to  break  out  the  endangered  aircraft.  The 
controller  mav  direct  the  breakout  earlier  if  he/she  believes  that  the  situation  requires  it. 

Compared  with  existing  systems,  the  PRM  improves  the  radar  update  interval  and  accuracy, 
provides  an  expanded  cross-track  display,  adds  color  to  the  display,  and  provides  a  10-s  track 
projection.  The  PRM  also  alerts  the  controller  of  a  possible  blunder  by  changing  the  color  of  the 
suspected  aircraft  from  green  to  yellow,  and  by  sounding  an  audible  alert  whenever  the  aircraft  is 
projected  to  enter  the  NTZ  within  10  s.  These  features  are  inextricably  linked  with  the  response  time 
of  the  controller.  The  controller  response  without  them  was  not  measured. 

Figure  3-6  diagrams  the  events  occurring  during  a  blunder  resolution.  One  measure  of  the 
effectiveness  of  the  system  is  the  alert  response  time  (ART).  Measurement  of  this  time  begins  when 
the  audible  Caution  Alert  sounds  and  ends  when  the  controller  begins  to  speak  the  breakout 
instruction  to  the  endangered  aircraft. 
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Figure  3-6.  Sequence  of  timing  events  from  start  of  approach  blunder  to  NTZ  penetration. 


Because  the  audible  alert  can  only  sound  when  the  target  display  is  updated  (at  whatever 
update  interval  is  being  tested),  ART,  taken  on  its  own,  tends  to  ignore  some  of  the  advantage  of  the 
faster  update  interval.  It  is,  therefore,  not  useful  as  a  single  measure  of  the  controller/radar 
effectiveness.  A  better  measure  is  how  much  in  advance  of  the  blundering  aircraft’s  penetration  of 
the  NTZ  the  Monitor  Controller  began  peaking.  This  was  termed  “net  controller  lead  time.”  This 
measures  the  combined  effects  of  (1)  the  sensor  update  interval,  (2)  the  performance  of  the  alerting 
algorithm,  and  (3)  the  interaction  of  the  controller  with  the  alert.  Alert  response  time  and  net 
controller  lead  time  are  illustrated  in  Figure  3-6. 

The  extraction  of  all  timing  data  relevant  to  controller  response  is  illustrated  in  Figure  3-7. 
The  upper  half  of  the  figure  depicts  the  extraction  of  ART.  The  audible  Caution  Alert  from  the 
approach  blunder  predictor,  the  controller's  instructions  to  pilots,  and  the  pilot  responses  were 
recorded  on  audio  tape,  along  with  a  time  code  that  provides  synchronization  to  the  digital  tape.  The 
audio  tapes  were  played  back  by  a  human  operator.  When  the  operator  heard  the  audible  Caution 
Alert  which  occurred  prior  to  an  approach  blunder,  he/she  pressed  a  key  on  the  keyboard  to  log  that 
time.  The  key  was  also  pressed  to  log  the  time  when  the  operator  heard  the  Monitor  Controller  begin 
the  breakout  instruction  to  the  endangered  aircraft.  A  computer  program  recorded  each  time  and  the 
difference  between  them.  Each  ART  was  measured  three  times,  and  the  averaged  value  was  used  in 
the  subsequent  data  analysis. 
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Audio  Data  Tracks 


Figure  3-7.  Controller  Response  Data  Extraction. 


The  lower  half  of  the  figure  depicts  the  extraction  of  all  other  pertinent  time  data.  All  track 
data  were  recorded  on  digital  tape  for  later  analysis.  The  relevant  times  for  each  approach  blunder 
included,  for  example,  the  beginning  of  the  turn,  the  time  of  the  Caution  Alert,  and  the  time  of  NTZ 
penetration. 

After  ART  data  were  extracted  from  the  audio  tapes,  they  were  stored  on  a  micro- VAX 
computer  and  managed  by  a  commercially-available  Relation  Data  Base  Management  System 
(RDBMS),  named  ORACLE  (Version  6.0).  Managing  the  data  with  an  RDBMS  allowed  the  analyst  to 
easily  and  directly  perform  queries  on  the  data  by  selecting  classes  of  information  and  isolating 
relationships  within  the  data. 

During  analysis,  the  data  were  stored  in  the  RDBMS  in  table  format  The  controller  response 
database  consists  of  three  tables:  (1)  the  BLUNDER  table  contains  a  summary  description  of  each  of 
the  approach  blunders,  (2)  the  CONTROLLER_RESPONSE  table  contains  the  measured  controller 
ARTs  for  each  of  the  tested  approach  blunders  described  in  the  BLUNDER  table,  and  (3)  the 
CAUTION_ALERT  table  contains  the  lead  time  (time  from  Caution  Alert  to  crossing  into  the  NTZ) 
for  each  approach  blunder.  The  analyst  used  all  three  tables  to  isolate  relationships  of  interest  and 
obtain  the  measured  data  for  analysis.  Appendices  H,  I,  and  J  list  examples  of  database  queries  and 
replies.  SPSS,  a  comprehensive,  integrated  system  for  statistical  data  analysis  [8],  was  used  to  analyze 
the  ART  data. 

As  discussed  in  Section  2,  the  major  design  of  the  studies  was  a  within-subject,  paired-sample 
design.  Therefore,  paired-sample  t-testing  was  used  to  assess  the  significance  of  the  difference 
between  mean  ARTs. 

The  ART  of  each  individual  under  one  condition  was  compared  to  the  ART  of  the  same 
individual  under  another  condition.  In  each  case,  only  one  variable  was  manipulated  in  each 
condition.  For  example,  in  the  case  of  sensor  update  interval,  the  ART  of  Controller  #1  in  condition 
1.0-s  sensor  update  interval,  “Single  Type”  blunder,  near  range,  calm  flight  path  was  compared  to 
the  ART  of  Controller  #1  in  condition  2.4-s  sensor  update  interval,  “Single  Type”  blunder,  near 
range,  calm  flight  path.  Thus,  the  comparison  is  of  the  same  controller's  performance  when  the  only 
variable  changed  was  the  sensor  update  interval;  the  other  variables  were  held  constant 

In  the  analysis  of  paired  data,  for  each  pair  of  cases,  the  difference  in  the  responses  is 
calculated.  The  statistical  procedure  used  tested  the  hypothesis  that  the  mean  difference  in  the 
population  is  zero.  The  statistic  used  was 


D  is  the  observed  difference  between  the  two  means  and  So  is  the  standard  deviation  of  the 
differences  of  the  paired  observations.  The  sampling  distribution  of  r,  if  the  differences  are  normally 
distributed  with  a  mean  of  zero,  is  Student’s  t  with  N- 1  degrees  of  freedom,  where  N  is  the  number  of 
pairs. 

The  test  result  provides  a  probability  value  that  tells  us  the  likelihood  that  the  difference 
between  means  is  attributable  to  chance.  In  general,  probability  values  of  .05  or  less  are  considered 
“significant.”  A  probability  value  of  .05  means  the  likelihood  of  this  effect  (the  difference  between 


means)  occurring  by  chance,  due  to  sampling  error,  is  5  in  100.  A  probability  criteria  of  .05  or  less 
means  that  we  will  reject  the  null  hypothesis,  i.e.,  reject  that  there  is  no  significant  difference  between 
means,  when  the  probability  value  is  .05  or  less. 

When  multiple  comparisons  are  made,  such  as,  performing  a  succession  of  within-subject 
t-tests,  each  result  on  its  own  is  well  founded.  However,  inherent  in  performing  a  succession  of 
analyses  is  the  fact  that  as  one  increases  the  number  of  comparisons  the  likelihood  of  finding 
“significance"  increases. 

In  an  attempt  to  be  conservative  in  interpreting  results,  an  adjustment  in  the  test  levels  can  be 
made.  This  adjustment  reflects  the  fact  that,  just  on  chance,  a  proportion  of  the  tests  will  be  flagged 
as  significant.  However,  one  should  be  aware  that  in  doing  so  we  will  err  on  the  conservative  side,  i.e., 
we  will  tend  to  screen  out  comparisons  that  really  are  significant.  One  way  to  make  this  adjustment  is 
to  use  the  Bonferroni  Multiple  Testing  Procedure  [9]. 

In  interpreting  the  results  of  the  within-subject  comparisons,  instead  of  using  a  per- 
comparison  probability  level  of  .05,  a  per-comparison  level  of  .05/C  is  used  (where  C  is  the  total 
number  of  planned  comparisons).  Use  of  this  procedure  causes  one  to  err  on  the  conservative  side, 
i.e.,  rejecting  probability  values  that  meet  the  criteria  of  .05  or  less  but  do  not  meet  the  new  criteria 
resulting  from  .05/C.  This  procedure  was  applied  to  the  probability  values  resulting  from  the  analysis 
of  within-subject  data  reported  for  Study  I  and  II.  Appendix  K  lists  the  number  of  within-subject 
t-test  comparisons  made  in  each  analysis  area  and  provides  the  adjusted  probability  criteria. 

As  discussed  in  Section  2,  in  studying  the  effects  of  the  variables  “controller  experience 
level”  and  “runway  separation,”  a  between-subject  independent-sample  design  was  used.  The 
statistic  used  was 

VSi2/A/1+S22//V2  (2) 

^1  is  the  sample  mean  of  Group  1,  Si2  is  the  variance,  and  is  the  sample  size. 

3.3.2  False  Breakout  Rate 

A  false  breakout  is  a  situation  in  which  a  controller  unnecessarily  initiated  a  breakout,  based 
on  the  behavior  of  an  aircraft  that  suggested  that  a  deviation  would  cause  penetration  of  the  NTZ, 
when  in  fact  it  remained  in  the  normal  operating  zone.  There  is  concern  that  the  presence  of  a 
yellow/audible  Caution  Alert  (predicting  NTZ  penetration  in  10  s  or  less  if  the  aircraft  remains  on  its 
present  course)  may  cause  an  unacceptable  number  of  false  breakouts.  At  worst,  an  excessive 
number  of  false  breakouts  could  offset  the  potential  capacity  gains  resulting  from  independent 
parallel  approaches. 

False  breakouts  were  studied  in  the  following  ways.  In  Study  I,  nuisance  Caution  Alerts  were 
generated  in  three  ways:  (1)  simulated  turbulence,  (2)  simulated  TNSE,  and  (3)  deviations  caused  by 
aircraft  that  were  scripted  to  be  distractions  to  the  controllers.  Scripted  distractions  were  situations  in 
which  all  aircraft  were  depicted  as  flying  routinely,  but  one  aircraft  suddenly  had  an  erratic  flight 
path  either  toward  or  away  from  the  NTZ.  In  Study  II,  nuisance  Caution  Alerts  were  generated  in  one 
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way,  i.e.,  simulated  TNSE.  In  both  studies,  data  were  collected  on  each  false  alert  breakout  resulting 
from  each  nuisance  Caution  Alert,  and  the  False  Breakout  Rates  were  calculated. 

3.3.3  Controller  Opinion 

Controller  Opinion  Survey  forms  were  developed  with  the  input  of  the  Core  Group  of 
Controllers.  The  surveys  for  Study  I  (Appendix  F)  and  Study  n  (Appendix  G)  were  used  to  obtain 
the  opinions  of  the  controllers  on  the  effectiveness  of  the  PRM  system  and  its  overall  acceptability  for 
use. 
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4.  RESULTS  OF  STUDY  I 


In  Section  4,  the  results  of  Study  I  are  presented.  Section  4.1  focuses  on  controller  response 
to  simulations  of  approach  blunders  presented  at  1.0-s  and  2.4-s  sensor  update  intervals.  3,400  ft 
runway  separation.  Section  4.2  focuses  on  controller  response  to  simulations  of  approach  blunders 
presented  at  4.8-s  sensor  update  interval,  3,400-ft  vs  4,300-ft  runway  separation. 

In  the  above  mentioned  sections,  the  effects  of  the  following  variables  on  the  Alert  Response 
Time  (ART)  of  controllers  are  reported  and  interpreted:  sensor  update  interval,  deviation  angle, 
aircraft  range  from  the  runway  threshold,  flight  path  conditions,  type  of  approach  blunder,  runway 
separation  (in  the  case  of  4.8-s  sensor  update  interval),  and  controller  experience  level.  These 
variables  were  defined  in  Section  2.1.1. 

Throughout  Section  4,  there  is  a  brief  description  of  each  analysis  performed.  ART  results 
which  are  statistically  significant  are  underlined.  Throughout  Sections  4.1.1  through  4.1.6,  4.2.1  and 
4.2.2,  and  4.4.1  through  4.4.4,  within-subject  comparisons  are  made.  The  probability  criteria  set  for 
determining  a  significant  difference  between  means  is  a  probability  value  of  .001  or  less 
(computation  of  probability  criteria  is  discussed  in  Section  3.3.1).  Section  4.1.7  and  4.3  document 
between-subject  comparisons  made.  The  probability  criteria  set  for  determining  a  significant 
difference  between  means  is  a  probability  value  of  .05  or  less. 

False  Breakout  results  are  discussed  when  applicable.  Results  of  the  Controller  Opinion 
Survey  used  for  Study  I  are  presented  in  Section  4.5. 

Before  delving  into  the  details  of  each  analysis  area,  an  “Overview  of  Findings”  is  presented. 
This  gives  the  reader  a  quick  look  at  the  answers  to  the  research  questions  (listed  in  Section  2.1.2) 
asked  in  Study  I.  Reference  is  made  to  the  section  in  which  greater  detail  on  findings  can  be  found. 
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Overview  of  Findings 

Question  1:  How  does  sensor  update  interval  affect  reaction  time?  This  will  give  us  information 
needed  to  decide  whether  a  1.0-s  or  2.4-s  sensor  update  interval  is  required  for  system 
effectiveness  at  3,400  ft  runway  separation. 

Findings:  Whether  the  sensor  update  interval  was  1.0-s  or  2.4-s,  mean  ART  did  not  differ 

significantly,  when  looking  at  all  “Single  Type”  approach  blunders  combined,  i.e., 
2.7  s  and  2.8  s,  respectively.  In  either  case,  on  average,  the  controllers  began  the 
breakout  instruction  before  the  blunderer  penetrated  the  NTZ.  However,  while  the 
mean  ART  of  both  update  intervals  does  not  differ  significantly,  the  faster  update 
interval  does  provide  increased  advance  warning  that  translates  into  a  greater  miss 
distance  between  aircraft.  (For  details,  see  Section  4.1.1.)  In  addition,  the  faster  sensor 

_ update  interval  resulted  in  a  lower  false  breakout  rate.  (For  details,  see  Section  4.1.8.) 

Question  2:  When  the  runway  separation  is  3,400  ft,  are  there  differences  in  reaction  time 
attributable  to: 

a.  the  angle  of  the  deviation  of  the  approach  blunder, 

b.  the  range  (nmi  from  the  runway  threshold)  of  aircraft  at  time  of  blunder, 

c.  the  flight  path  condition  during  the  approach  blunder, 

d.  the  type  of  approach  blunder,  for  example,  one  involving  one  aircraft  at  a  fast 
speed  deviating  toward  another  aircraft  at  a  slow  speed,  and 

e.  controller  experience  level? 

Findings:  a.  Yes,  whether  the  sensor  update  interval  was  1.0  s  or  2.4  s,  mean  ART  differed 

significantly  due  to  deviation  angle.  The  mean  ART  for  30-deg  approach 
blunders  was  approximately  1.0  s  faster  than  for  15-deg  approach  blunders.  (For 
details,  see  Section  4.1.3.). 

b.  Yes,  when  comparing  approach  blunders  occurring  at  far  range  vs  near  range, 
15-deg  deviation  angle:  whether  an  approach  blunder  was  presented  at  1.0-s  or 
2.4-s  sensor  update  interval,  the  mean  ART  was  approximately  1.0  s  slower  for 
approach  blunders  at  the  far  range.  However,  there  was  no  significant  difference 
in  ART  for  30-deg  approach  blunders,  when  comparing  far  vs  near  ranges.  (For 
details,  see  Section  4.1.4.) 

c.  No,  for  approach  blunders  with  30-deg  deviations,  no  significant  difference  was 
found  in  mean  ART  when  comparing  blunders  which  occurred  in  calm  vs 
turbulent  flight  path  conditions.  (For  details,  see  Section  4.1.5.) 

d.  No,  in  general,  mean  ART  did  not  appear  to  vary  greatly  as  a  function  of 
approach  blunder  type.  (For  details,  see  Section  4.1.6.) 

e.  Yes,  experienced  Monitor  Controllers  responded  with  a  mean  ART  of 
approximately  1.0  s  slower  than  the  controllers  who  had  no  previous  experience 

_ as  Monitor  Controllers.  (For  details,  see  Section  4.1.7.)  _ 
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Question  3:  Are  there  differences  in  reaction  time  attributable  to  runway  separation  in  feet  from 
centerline  to  centerline?  The  specific  case  tested  was  the  presentation  of  approach 
blunders  at  the  4.8-s  sensor  update  interval  with  3,400-ft  vs  4,300-ft  runway 
separation. 

Findings:  Yes,  mean  ART  was  significantly  faster  when  the  runway  separation  was  increased 

from  3,400  ft  to  4,300  ft.  The  mean  ART  in  the  4,300-ft  runway  separation  condition 
was  2.6  s  faster  than  the  mean  ART  in  the  3,400-ft  runway  separation  condition.  (For 

_ details,  see  Section  4.3.) _ 

Question  4:  When  runway  separation  is  4,300  ft  and  the  sensor  update  interval  is  4.8  s,  are  there 
differences  in  reaction  time  attributable  to:  a.  deviation  angle,  b.  range,  c.  flight  path 
condition,  and  d.  type  of  approach  blunder? 

Findings:  a.  No,  whether  the  deviation  angle  was  IS  deg  or  30  deg  the  mean  ART  did  not 

differ  significantly.  In  the  case  of  15-deg  deviation  angle,  mean  ART  was  0  s,  and 
in  the  case  of  30-deg  deviation  angle,  mean  ART  was  1.0  s.  (For  details,  see 
Section  4.4.1.) 

b.  Yes,  when  comparing  30-deg  deviation  angle,  occurring  at  far  range  vs  near 
range:  whether  an  approach  blunder  was  presented  at  1.0-s  or  2.4-s  sensor  update 
interval,  mean  ART  was  significantly  slower  (i.e.,  approximately  2.5  s  slower)  for 
approach  blunders  at  the  far  range.  In  the  15-deg  deviation  angle  condition, 
only  near  range  deviations  were  included.  Therefore,  no  comparison  could  be 
made  of  15-deg  approach  blunders,  near  vs  far  ranges.  (For  details,  see  Section 
4.4.2.) 

c.  No,  mean  ART  did  not  differ  significantly  when  comparing  calm  vs  turbulent 
flight  path  conditions.  In  both  calm  and  turbulent  conditions,  the  mean  ART  was 
0  s.  (For  details,  see  Section  4.4.3.) 

d.  No,  mean  ART  did  not  differ  significantly  when  comparing  “Single  Type”  vs 
“rast/Slow  Type”  approach  blunders  (15-deg  deviation  angle,  near  range).  In 

_ either  case,  the  mean  ART  was  approximately  0  s.  (For  details,  see  Section  4,4.4.) 

Question  5:  Will  controllers  accept  the  PRM  system  as  a  means  of  safely  conducting  independent 
parallel  approaches  during  IFR  conditions  to  parallel  runways  spaced  3,400  ft  apart? 

Findings:  Yes,  controllers  were  unanimous  in  their  enthusiasm  for  the  PRM  system.  They  all 

reported  that  with  the  PRM  system,  with  its  high  resolution  color  display  and 
automated  alerts,  they  could  monitor  the  final  better  than  with  the  current  ARTS 

_ (Automated  Terminal  Radar  System).  (For  details,  see  Section  4.5). _ 
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4.1  RESPONSES  TO  SIMULATIONS  OF  APPROACH  BLUNDERS  PRESENTED  AT  1.0-S 
AND  2.4-S  SENSOR  UPDATE  INTERVALS,  3, 400- FT  RUNWAY  SEPARATION 


4.1.1  Overall  Effect  of  Sensor  Update  Interval 

In  order  to  assess  the  effects  of  the  1.0-s  and  2.4-s  sensor  update  intervals  on  ART,  the 
responses  of  the  controllers  to  the  eight  “Single  Type”  approach  blunders  presented  at  each  of  the 
respective  sensor  update  intervals  were  analyzed.  Results  from  the  “Fast/Slow  Type”  and 
“Distraction  Type”  blunders  are  reviewed  in  Section  4.1.6.  They  are  reviewed  separately  because  of 
the  unique  characteristics  of  these  approach  blunders. 

ART  results  are  presented  in  Table  4-1  for  the  1.0-s  and  2.4-s  sensor  update  intervals.  For 
each  sensor  update  interval,  the  mean  ART,  standard  deviation  (sd),  and  minimum  and  maximum 
ART  values  are  reported. 

Table  4-1 

A  Comparison  of  ART  During  Simulations  of  Simultaneous 
Parallel  Approaches  to  Runways  with  3,400-ft  Separation 


Update  Interval 

1.0  S 

2.4  S 

#  of  Responses 

392 

392 

Maximum  ART 

16.3 

11.5 

Minimum  ART 

-2.8 

-2.7 

sd 

1.9 

1.9 

Mean  ART 

2.7 

2.8 

T-test  results  indicate  t  =  -1.02,  P  =.309  (degrees  of  freedom  (df)  =  391).  Since  the 
probability  that  the  difference  in  mean  ART  is  >.001,  the  null  hypothesis  is  accepted,  i.e.,  no 
significant  difference  was  found  between  the  mean  ART  of  1.0-s  vs  2.4-s  sensor  update  interval.  The 
conclusion  that  follows  is  that  regardless  of  the  speed  at  which  the  information  was  provided  to  the 
controller  (limited  to  the  two  update  intervals  tested),  the  individual  controller's  mean  ART  did  not 
change  significantly. 

Although  the  use  of  these  two  sensor  update  intervals  did  not  cause  a  significant  change  in 
mean  ART,  there  are  two  additional  factors  to  consider.  These  factors  are  (1)  the  amount  of 
information,  provided  by  each  of  the  sensor  update  intervals,  on  which  the  controller  can  base  the 
breakout  decision,  and  (2)  the  Net  Controller  Lead  Time  (defined  in  Section  3.3.1). 

Let  us  consider  the  amount  and  sequence  of  information  provided  by  each  of  the  two  sensor 
update  intervals.  This  is  graphically  represented  in  Figures  4- la  and  b.  Looking  at  Table  4-1,  we  see 
that  on  average  in  the  1.0-s  sensor  update  interval  condition,  controllers  responded  at  2.7  s.  As 
shown  in  Figure  4- la,  a  Caution  Alert  is  received  at  0-s  elapsed  time,  a  revised  aircraft  position  is  seen 
at  1.0-s  elapsed  time,  and  another  revised  aircraft  position  is  seen  at  2.0-s  elapsed  time.  Then,  on 
average,  the  controller  made  a  decision  to  break  out  the  endangered  aircraft  at  approximately  2.7-s 
elapsed  time.  Therefore,  the  decision  was  generally  based  on  information  from  three  sequential 
sensor  updates. 
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On  average  in  the  2.4-s  sensor  update  interval  condition,  controllers  responded  at  2.8  s.  As 
shown  in  Figure  4- lb,  a  Caution  Alert  is  received  at  0-s  elapsed  time  and  one  revised  aircraft  position 
is  seen  at  2.4-s  elapsed  time.  Therefore,  in  the  2.4-s  sensor  update  interval  condition,  on  average,  the 
controller’s  decision  was  based  on  information  from  two  sequential  sensor  updates,  while  in  the  1.0-s 
sensor  update  interval  condition,  on  average,  the  controller’s  decision  was  based  on  information  from 
three  sequential  sensor  updates. 

Although  an  extra  update  was  obtained  with  the  more  frequent  sensor  update  interval,  data 
show  that  it  still  took  almost  3.0  s  to  make  the  decision.  It  appears  that  regardless  of  how  quickly  the 
information  was  presented  (within  the  confines  of  3,400-ft  runway  separation,  1.0-s  vs  2.4-s  sensor 
update  interval),  there  is  a  limit  to  how  quickly  the  human  operator  can  perform  the  task  at  hand. 
The  following  steps  are  needed  in  taking  corrective  action,  i.e.,  initiating  the  breakout  communication 
to  the  endangered  aircraft;  perceive  the  alert,  analyze  the  information  presented,  decide  to  take  action, 
and  then  begin  to  initiate  a  verbal  response. 


Update  of  Aircraft  Position:  - 

Elasped  Time:  0  s 

— *•© - ►© 

1  s  2s 

2.7  s 

Controller  Action:  watch 

watch  watch 

breakout 

communication 

Figure  4-la.  Sequence  of  Timing  Events  during  1.0-s  Sensor  Update  Interval. 

Update  of  Aircraft  Position:  - 

■© 

Elasped  Time:  0  s 

2.4  s 

2.8  s 

Controller  Action:  watch 

watch 

breakout 

communication 

Figure  4-lb.  Sequence  of  Timing  Events  during  2.4-s  Sensor  Update  Interval. 

In  the  case  of  the  2.4-s  sensor  update  interval  condition,  if  the  controllers  had  waited  for  the 
“Aircraft  Position  #3”  (as  was  the  case  in  the  1.0-s  sensor  update  interval  condition),  the  controllers 
would  have  had  to  wait  until  4.8  s  had  elapsed.  It  is  obvious  that  the  controllers  judged  that  it  would 
be  too  long  to  wait  before  taking  action.  Perhaps  they  would  have  liked  another  update,  but 
considering  the  severity  of  the  situation,  i.e.,  impending  penetration  of  the  NTZ  and  endangering 
another  aircraft,  they  chose  not  to  wait. 
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4.1.2  Net  Controller  Lead  Time 


Let  us  consider  the  Net  Controller  Lead  Time  provided  by  the  sensor  update  intervals.  Table 
4-2  presents  mean  values  of  net  controller  lead  time.  The  values  are  a  measure  of  how  much  in 
advance  of  the  blundering  aircraft  entering  the  NTZ  the  controller  broke  out  the  endangered  aircraft. 
Net  Controller  Lead  Time  is  depicted  in  Figure  3-6.  It  is  calculated  by  subtracting  the  controller 
ART  from  the  Caution  Alert  lead  time  (CALT).  CALT  is  the  time  between  the  first  PRM  Caution 
Alert  during  a  blunder  and  when  the  blundering  aircraft  enters  the  NTZ.  For  the  study,  CALT  was 
measured  from  digital  recordings  of  the  blundering  aircraft  Given  that  the  blundering  aircraft  went 
on  to  enter  the  NTZ  and  continued  to  endanger  the  other  aircraft,  increases  in  controller  lead  time 
translate  directly  to  more  time  for  the  endangered  aircraft  to  maneuver  clear.  The  time  is  a 
reasonable  measure  of  how  much  advance  warning  the  system  (radar,  displays,  predictors,  controller 
training,  and  procedures)  gives  to  the  endangered  aircraft  The  faster  update  intervals  give  earlier 
warnings.  Table  4-2  shows  the  mean  values  of  all  the  times  shown  in  Figure  3-6. 

Table  4-2 


Net  Controller  Response  Lead  Time 
Mean  Seconds  Prior  to  NTZ  Penetration  at  3,400-ft  Runway  Separation 
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In  summary,  it  appears  that  regardless  of  sensor  update  interval,  the  mean  ART  of  controllers 
remained  basically  the  same.  However,  the  1.0-s  sensor  update  condition  did  provide  one  additional 
update  of  the  aircraft  position.  It  is  not  know  if  this  additional  update  made  the  controllers  feel  more 
confident  in  their  decision  to  breakout  the  endangered  aircraft.  The  advantage  of  the  faster  update 
interval  is  the  increased  amount  of  advance  warning  given  by  the  1.0-s  vs  2.4-s  sensor  update  interval. 
When  the  ART  in  response  to  both  update  intervals  is  similar,  but  the  faster  update  interval  provides 
increased  advance  warning,  this  translates  into  a  greater  miss  distance  (approximately  120  ft  per 
second)  between  aircraft  when  the  faster  update  interval  is  used. 

4.1.3  Effect  of  Deviation  Angle 

In  order  to  assess  the  effect  of  deviation  angle  in  the  1.0-s  and  2.4-s  sensor  update  interval 
conditions,  the  responses  of  the  controllers  to  the  eight  “Single  Type”  approach  blunders  presented 
at  each  of  the  respective  sensor  update  intervals  were  analyzed. 

ART  results  are  presented  in  Table  4-3  for  approach  blunders  with  both  a  15-deg  and  30-deg 
deviation  angle.  The  “Fast/Slow  Type”  and  “Distraction  Type”  approach  blunders  were  omitted 
from  this  table  since  these  two  types  of  blunders  did  not  include  all  the  combinations  of  blunder 
angles,  ranges,  and  flight  path  conditions  that  were  included  in  the  “Single  Type”  of  approach 


blunders.  The  “Single  Type”  conditions  represent  a  full  data  set,  allowing  for  a  balanced 
comparison  of  15-deg  vs  30-deg  deviation  angle;  i.e.,  every  approach  blunder  tested  in  “Single 
Type,"  15-deg  deviation  angle  was  also  tested  in  “Single  Type”  30-deg  deviation. 


Table  4-3 

A  Comparison  of  ART  for  “Single  Type"  Approach  Blunders 
15-deg  vs  30-deg  Deviation  Angle  (3,400-ft  Runway  Separation) 
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Results  indicate  that  at  both  sensor  update  intervals,  the  mean  ART  for  approach  blunders 
with  a  30-deg  deviation  angle  are  approximately  1.0  s  faster  than  approach  blunders  with  15-deg 
deviation  angle.  With  a  30-deg  deviation  angle,  penetration  of  the  NTZ  occurred  more  rapidly  than 
with  a  15-deg  deviation  angle.  There  was  less  time  before  NTZ  penetration;  therefore,  the  controller 
reacted  more  rapidly. 

Two  paired-sample  t-tests  were  performed  to  assess  the  effect  of  ART  attributable  to  the 
deviation  angle,  within  each  of  the  sensor  update  interval  conditions; 

(1)  1.0-s  sensor  update  interval,  15-deg  vs  30-deg  deviation  angle: 

Results  indicate  that  t  =  6.81,  P  =  .000  (df=  197).  Since  the  probability  is  <.001,  the 
null  hypothesis  is  rejected;  i.e.,  there  is  a  significant  difference  in  mean  ART  due  to 
deviation  angle.  This  is  a  very  strong  effect.  The  probability  of  this  effect  occurring 
by  chance  due  to  sampling  error  is  less  than  1  in  1,000. 

(2)  2.4-s  sensor  update  interval,  15-deg  and  30-deg  deviation  angle; 

Results  indicate  that  t  =  6.10,  P  =  .000  (df  =  193).  Like  the  1.0-s  sensor  update  interval 
condition,  the  2.4-s  sensor  update  interval  condition  probability  is  <.001,  and  the  null 
hypothesis  is  rejected.  Therefore,  there  is  a  significant  difference  in  mean  ART  due  to 
deviation  angle.  This  too  is  a  very  strong  effect.  For  both  sensor  update  intervals  the 
probability  of  this  effect  occurring  by  chance  due  to  sampling  error  is  less  than  1  in 
1,000. 

The  results  indicate  that  in  both  sensor  update  interval  conditions  (1.0  s  and  2.4  s),  the  angle 
of  the  deviation  was  shown  to  make  a  significant  difference  in  the  mean  ART.  Controllers  responded 
to  approach  blunders  with  a  30-deg  deviation  angle  with  greater  speed  (approximately  1.0  s  faster) 
than  a  15-deg  deviation  angle. 

The  reason  for  this  additional  speed  in  a  response  to  a  30-deg  blunder  is  probably  two-fold. 
First,  a  30-deg  blunder  is  quite  severe  and  requires  immediate  action,  while  with  a  15-deg  blunder,  the 
controller  can  wait  a  little  longer  to  see  how  the  situation  develops  and  then  make  a  decision  on 
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whether  or  not  a  breakout  is  necessary.  Second,  the  expanded  view,  used  to  enable  quick  detection  of 
deviations,  exaggerates  the  angle  of  the  deviation.  This  causes  a  30-deg  deviation  to  appear  to  be 
even  more  severe,  resulting  in  increased  speed  of  response. 

4.1.4  Effect  of  Approach  Blunder  Range 

The  question  arises:  Since  the  results  on  deviation  angle  indicate  that  controllers  responded  to 
approach  blunders  with  a  30-deg  deviation  angle  approximately  1.0  s  faster  than  with  a  15-deg 
deviation  angle,  then  how  does  the  added  variable  of  the  range  at  which  the  approach  blunder 
occurred  affect  ART?  For  example,  is  the  ART  to  all  approach  blunders  with  a  15-deg  deviation 
angle  approximately  3.0  s  or  does  the  ART  vary  with  the  range  at  which  the  approach  blunder 
occurs? 

ART  results  are  presented  in  Table  4-4  for  approach  blunders  with  a  15-deg  and  30-deg 
deviation  angles  when  the  range  of  the  approach  blunder  was  near  (2  to  4  rani)  vs  far  (8  to  12  nmi) 
from  the  runway  threshold.  ART  for  both  1.0-s  and  2.4-s  sensor  update  intervals  in  the  “Single 
Type”  of  approach  blunder  are  shown.  Again,  “Fast/Slow  Type”  and  “Distraction  Type”  approach 
blunders  were  omitted. 

Table  4-4 


A  Comparison  of  ART  for  “Single  Type”  Approach  Blunders, 

15-deg  vs  30-deg  Approach  Blunder  Angle,  and  Approach  Blunder  Ranges 
of  Near  vs  Far  (3,400-ft  Runway  Separation) 
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Four  paired-sample  t-tests  were  performed  to  assess  the  effect  of  range  on  the  mean  ART  to 
approach  blunders  within  15-deg  and  30-deg  deviation  angles. 

(1)  1.0-s  sensor  update  interval,  15-deg  deviation  angle,  near  vs  far  range: 

Results  indicate  that  t  =  -3.59,  P  =  .001  (d/=  98).  Since  the  probability  is  .001,  the  null 
hypothesis  is  rejected;  i.e.,  there  is  a  significant  difference  in  mean  ART  due  to  the 
ranee  at  which  the  approach  blunder  with  a  15-deg  deviation  angle  occurred  during  the 
1.0-s  sensor  update  interval.  This  is  a  very  strong  effect  The  probability  of  this  effect 
occurring  by  chance  due  to  sampling  error  is  approximately  1  in  1,000.  In  the  far 
range  approach  blunders,  the  ART  was  approximately  0.8  s  longer  than  during  the  near 
range  approach  blunders. 
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(2)  1.0-s  sensor  update  interval.  30-deg  deviation  angle,  near  vs  far  range: 

Results  indicate  that  t  =  -1.48,  P  =  .143  (df  =  98).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
the  range  at  which  the  approach  blunder  with  a  30-deg  deviation  angle  occurred  during 
the  1.0-s  sensor  update  interval. 

(3)  2.4-s  sensor  update  interval,  15-deg  deviation  angle,  near  vs  far  range: 

Results  indicate  that  t  =  -4.42,  P  =  .000  (df  -  98).  Since  the  probability  is  <.001,  the 
null  hypothesis  is  rejected;  i.e.,  there  is  a  significant  difference  in  mean  ART  due  to  the 
ranee  at  which  the  approach  blunder  with  a  15-deg  deviation  angle  occurred  during  the 
2.4-s  sensor  update  interval.  This  is  a  veiy  strong  effect  The  probability  of  this  effect 
occurring  by  chance  due  to  sampling  error  is  less  than  1  in  1,000.  This  result  is 
compatible  with  the  results  found  for  the  same  deviation  angle  and  range  in  the  1.0-s 
sensor  update  interval  condition. 

(4)  2.4-s  sensor  update  interval,  30-deg  deviation  angle,  near  vs  far  range: 

Results  indicate  that  t  =  .56,  P  =  .579  (df  =  95) .  Since  the  probability  is  >.001,  the  null 
hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to  the 
range  at  which  the  approach  blunders  with  a  30-deg  deviation  angle  occurred  during 
the  2.4-s  sensor  update  interval. 


In  summary,  whether  an  approach  blunder  was  presented  at  1.0-s  or  2.4-s  sensor  update 
interval,  when  the  deviation  angle  was  15  deg,  the  ART  was  significantly  slower  for  approach 
blunders  at  far  range  than  at  near  range.  It  may  be  that  this  effect  occurred  in  the  approach  blunders 
with  a  15-deg  deviation  angle  and  not  in  the  30-deg  deviation  angle,  since  with  the  more  severe 
deviation  angle  of  30  deg,  the  controllers  know  that  they  must  act  as  quickly  as  possible  regardless  of 
the  range  of  the  blunder.  A  question  is:  Why  on  the  15-deg  approach  blunders  would  ART  be  slower 
when  the  approach  blunder  occurs  further  from  tire  runway  threshold?  It  may  be  because  at  the  far 
range  the  aircraft  are  not  stable  on  the  localizer,  TNSE  is  greater,  and  more  Caution  Alerts  are 
occurring.  The  controllers  may  have  been  waiting  for  greater  flight  path  stabilization  and,  therefore, 
waited  longer  after  the  Caution  Alert  was  received  at  the  far  range  vs  near  range. 

4.1.5  Effect  of  Flight  Path  Condition 

The  next  area  analyzed  was  the  effect  of  the  simulated  flight  path  condition  on  ART  during 
the  presentation  of  approach  blunders  at  the  two  sensor  update  intervals.  Since  it  was  found  that  the 
responses  to  approach  blunders  with  a  30-deg  deviation  angle  did  not  differ  significantly  by  range 
(see  Section  4.1.4),  near  and  far  range  30-deg  deviation  blunders  were  pooled,  and  the  effect  of  calm 
vs  turbulent  flight  path  were  analyzed. 

ART  results  are  presented  in  Table  4-5  for  approach  blunders  with  30-deg  deviation  angle, 
calm  vs  turbulent  flight  path  conditions.  ART  for  both  1.0-s  and  2.4-s  sensor  update  intervals  in  the 
“Single  Type”  approach  blunder  are  shown. 


Table  4-5 


A  Comparison  of  ART  for  “Single  Type”  Approach  Blunders, 
30-deg  Deviation  Angle,  Calm  vs  Turbulent  Flight  Path  Conditions 
(3,400-ft  Runway  Separation) 
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Two  paired-sample  t-tests  were  performed  to  assess  the  effect  of  flight  path  condition  within 
ART  to  approach  blunders  with  30-deg  deviation  angle. 

(1)  1.0-s  sensor  update  interval,  30-deg  deviation  angle,  calm  vs  turbulent  flight  path 
conditions: 

Results  indicate  that  t  =  -.66,  P  =  .513  (df  =  98).  Since  the  probability  is  >.001,  the  null 
hypothesis  is  accepted,  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to  the 
flight  path  condition  seen  in  the  approach  blunders  with  30-deg  deviation  angle  which 
occurred  during  the  1.0-s  sensor  update  interval. 

(2)  2.4-s  sensor  update  interval.  30-deg  deviation  angle,  calm  vs  turbulent  flight  path 
conditions: 

Results  indicate  that  t  =  -.37,  P  =  .715  (df  =  95).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted,  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
the  flight  path  condition  seen  in  the  approach  blunders  with  30-deg  deviation  angle 
which  occurred  during  the  2.4-s  sensor  update  interval. 

In  summary,  whether  the  sensor  update  interval  was  1.0  s  or  2.4  s,  there  was  no  significant 
difference  in  mean  ART,  and  we  find  that  there  is  no  significant  difference  in  mean  ART  attributed  to 
flight  path  condition  in  the  approach  blunders  with  a  30-deg  deviation  angle. 

Since  it  was  found  that  the  responses  to  approach  blunders  with  a  15-deg  deviation  angle 
differed  significantly  by  range  (see  Section  4.1.4),  approach  blunders  with  near  and  far  range, 
15-deg  deviation  angle,  were  analyzed  separately  to  determine  the  effects  of  calm  vs  turbulent  flight 
path.  The  values  within  this  grouping  are  shown  in  Table  4-6. 
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Table  4-6 


A  Comparison  of  ART  for  “Single  Type”  Approach  Blunders, 
15-deg  Deviation  Angle,  Near  and  Far  Ranges,  and  Calm  vs  Turbulent 
Flight  Path  Conditions  (3,400-ft  Runway  Separation) 
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Four  paired-sample  t-tests  were  performed  to  assess  the  effect  of  flight  path  condition  within 
ART  to  approach  blunders  with  a  15-deg  deviation  angle,  sorted  by  range. 

(1)  1.0-s  sensor  update  interval,  15-deg  deviation  angle,  near  range,  calm  vs  turbulent  flight 
path  conditions: 

Results  indicate  that  t  =  .43,  P  =  .670  {df  =  48).  Since  the  probability  is  >.001,  the  null 
hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to  the 
flight  path  condition  seen  in  the  approach  blunders  with  15-deg  deviation  angle  which 
occurred  at  near  range  during  the  1.0-s  sensor  update  interval. 

(2)  1.0-s  sensor  update  interval,  15-deg  deviation  angle,  far  range,  calm  vs  turbulent  flight 
path  conditions: 

Results  indicate  that  t  =  2.98,  P  =  .005  ( df  -  49).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
the  flight  path  condition  seen  in  the  approach  blunders  with  15-deg  deviation  angle 
which  occurred  at  the  far  range  during  the  1.0-s  sensor  update  interval. 

(3)  2.4-s  sensor  update  interval,  15-deg  deviation  angle,  near  range,  calm  vs  turbulent  flight 
path  conditions. 

Results  indicate  that  t  =  -.47,  P  =  .639  {df  -  48).  Since  the  probability  is  >.001,  the  null 
hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to  the 
flight  path  condition  seen  in  the  approach  blunders  with  a  15-deg  deviation  angle  which 
occurred  at  the  near  range  during  the  2.4-s  sensor  update  interval. 

(4)  2.4-s  sensor  update  interval,  15-deg  deviation  angle,  far  range,  calm  vs  turbulent  flight 
path  conditions: 

Results  indicate  that  t  =  -1.18,  P  =  .242  {df  =  48).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
the  flight  path  condition  seen  in  the  approach  blunders  of  a  15-deg  approach  deviation 
angle  which  occurred  at  the  far  range  during  the  2.4-s  sensor  update  interval. 

In  summary,  when  comparing  calm  vs  turbulent  conditions,  15-deg  deviation  angle,  no 
significant  difference  in  mean  ART  was  found. 
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4.1.6  Effect  of  Approach  Blunder  Type 


Various  analyses  were  performed  in  order  to  assess  the  difference  in  ART  which  may  have 
been  attributed  to  the  approach  blunder  type.  Three  blunder  types  were  studied:  “Single  Type.” 
“Distraction  Type"  (a  Single  Blunder  preceded  by  a  distraction),  and  a  “Fast/Slow  Type”  approach 
blunder  (see  Section  2.1.1  for  approach  blunder  definitions).  The  “Simultaneous  Missed  Approach 
Type”  failed  to  develop  useful  information.  Results  from  these  approach  blunders  are,  therefore,  not 
included  in  the  data  analysis.  This  type  of  approach  blunder  involved  a  blunder  after  a  dual  missed 
approach.  Since  most  scenarios  ended  with  both  aircraft  landing  or  with  one  aircraft  blundering  and 
the  other  breaking  out,  controllers  witnessing  a  dual  missed  approach  -  a  rare  event  in  actual 
operations  -  acted  to  resolve  any  possibility  of  a  missed  approach  blunder  by  turning  the  aircraft 
away  from  each  other  even  before  a  Caution  Alert  occurred.  This  phenomenon  was  found  to  occur 
in  both  live  flights  and  simulations. 

In  this  section,  ART  results  on  “Single  Type”  vs  “Fast/Slow  Type”  approach  blunders, 
3,400-ft  runway  separation,  are  presented  in  Tables  4-7  through  4-10.  ART  results  on  “Single 
Type”  vs  “Distraction  Type”  approach  blunders  are  presented  in  Tables  4-11  through  4-14. 
Following  each  table,  results  are  presented  regarding  significant  difference  in  mean  ART  to 
attributable  approach  blunder  type. 

Four  paired-sample  t-tests  were  performed  in  order  to  compare  “Single  Type”  vs  “Fast/Slow 
Type”  approach  blunders. 

(1)  1.0-s  sensor  update  interval,  lS-deg  deviation  angle,  near  range,  calm  flight  path 

condition,  “Single  Type”  vs  “Fast/Slow  Type”: 

Table  4-7 

A  Comparison  of  ART  for  “Single  Type”  vs  "Fast/Slow  Type”  Approach  Blunders, 
1.0-s  Sensor  Update  Interval,  15-deg  Deviation  Angle,  Near  Range, 

Calm  Flight  Path  Condition 
(3,400-ft  Runway  Separation) 


Blunder  Type 

“Slnaie” 

#  of  Responses 

49 

49 

Maximum  ART 

9.8 

12.9 

Minimum  ART 

-2.6 

-1.7 

sd 

2.3 

2.9 

Mean  ART 

2.7 

2.9 

Results  indicate  that  t  =  -.52,  P  =  .605  {df  =  48).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
blunder  type. 
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(2)  2.4-s  sensor  update  interval,  15-deg  deviation  angle,  near  range,  calm  flight  path 

condition,  “Single  Type”  vs  “Fast/Slow  Type”: 

Table  4-8 

A  Comparison  of  ART  for  “Single  Type”  vs  “Fast/Slow  Type”  Approach  Blunders, 
2.4-s  Sensor  Update  Interval,  15-deg  Deviation  Angie,  Near  Range, 

Calm  Flight  Path  Condition 
(3,400-ft  Runway  Separation) 


Blunder  Tvoe 

“Slnale” 

“Fast/Slow" 

#  of  Responses 

47 

47 

Maximum  ART 

9.1 

8.0 

Minimum  ART 

-2.4 

-2.6 

sd 

2.3 

2.4 

Mean  ART 

2.6 

2.0 

Results  indicate  that  t  =  1.30,  P  =  .201  (df  =  46).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
blunder  type. 

(3)  1.0-s  sensor  update  interval,  15-deg  deviation  angle,  far  range,  calm  flight  path 

condition,  “Single  Type”  vs  “Fast/Slow  Type”: 

Table  4-9 

A  Comparison  of  ART  for  “Single  Type”  vs  “Fast/Slow  Type”  Approach  Blunders, 
1.0-s  Sensor  Update  Interval,  15-deg  Deviation  Angle,  Far  Range, 

Calm  Flight  Path  Condition 
(3,400-ft  Runway  Separation) 


Blunder  Type 

“Slnale” 

■HtfflKU:  L4VE1  ITTSHH 

#  of  Responses 

50 

50 

Maximum  ART 

8.5 

16.3 

Minimum  ART 

0.5 

-0.6 

sd 

2.3 

3.3 

Mean  ART 

3.9 

4.9 

Results  indicate  that  t  =  -1.79,  P  =  .079  (df  =  49).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
blunder  type. 
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(4)  2.4-s  sensor  update  interval,  15-deg  deviation  angle,  far  range,  calm  flight  path 

condition,  “Single  Type”  vs  “Fast/Slow  Type”: 


Table  4-10 


A  Comparison  of  ART  for  “Single  Type”  vs  “Fast/Slow  Type”  Approach  Blunders, 
2.4-s  Sensor  Update  Interval,  15-deg  Deviation  Angle,  Far  Range, 

Calm  Flight  Path  Condition 
(3,400-ft  Runway  Separation) 


Blunder  Tvoe 

“Slnale” 

IMBJ 1 T  1 1k\ 

#  of  Responses 

49 

49 

Maximum  ART 

11.5 

8.2 

Minimum  ART 

0.2 

1.2 

sd 

2.4 

1.6 

Mean  ART 

3.7 

3.8 

Results  indicate  that  t  =  -.34,  P  =  .738  (df=  48).  Since  the  probability  is  >.001,  the  null 
hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
blunder  type. 

In  summary,  whether  the  approach  blunder  type  was  “Single  Type”  or  “Fast/Slow  Type,” 
when  comparing  mean  ART  from  approach  blunders  presented  (holding  constant  the  same  sensor 
update  interval,  blunder  angle,  range,  and  flight  path  condition)  there  is  no  significant  difference  in 
the  mean  ART  due  to  approach  blunder  type. 

Four  paired-sample  t-tests  were  performed  in  order  to  compare  “Single  Type”  vs 
“Distraction  Type”  approach  blunders. 

(1)  1.0-s  sensor  update  interval,  15-deg  deviation  angle,  far  range,  calm  flight  path 

condition,  “Single  Type”  vs  “Distraction  Type”: 

Table  4-11 

A  Comparison  of  ART  for  “Single  Type"  vs  “Distraction  Type”  Approach  Blunders, 
1.0-s  Sensor  Update  Interval,  15-deg  Deviation  Angle,  Far  Range, 

Calm  Flight  Path  Condition 
(3,40tf-ft  Runway  Separation) 


Blunder  Tvoe 

“Slnale" 

“Distraction” 

#  of  Responses 

49 

49 

Maximum  ART 

8.5 

10.0 

Minimum  ART 

0.5 

-2.8 

sd 

2.3 

2.8 

Mean  ART 

4.0 

3.2 

Results  indicate  that  t  =  1.85,  P  -  .070  {df  =  48).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
blunder  type. 
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(2)  2.4-s  sensor  update  interval,  15-deg  deviation  angle,  far  range,  calm  flight  path 

condition,  “Single  Type”  vs  “Distraction  Type”: 

Table  4-12 

A  Comparison  of  ART  for  “Single  Type”  vs  “Distraction  Type”  Approach  Blunders, 
2.4-s  Sensor  Update  Interval,  15-deg  Deviation  Angle,  Far  Range, 

Calm  Flight  Path  Condition 
(3,400-ft  Runway  Separation) 


Results  indicate  that  t  =  3.07,  P  =  .003  (df  =  48).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
blunder  type. 

(3)  1.0-s  sensor  update  interval,  30-deg  deviation  angle,  far  range,  calm  flight  path 

condition,  “Single  Type”  vs  “Distraction  Type”: 

Table  4-13 

A  Comparison  of  ART  for  “Single  Type”  vs  “Distraction  Type” 

Approach  Blunders,  1.0-s  Sensor  Update  Interval,  30-deg  Deviation  Angle, 

Far  Range,  Calm  Flight  Path  Condition 
(3,400-ft  Runway  Separation) 


Results  indicate  that  t-  -1.81,  P  =  .077  {df  =  48).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
blunder  type. 
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(4)  2.4-s  sensor  update  interval,  30-deg  deviation  angle,  far  range,  calm  flight  path 

condition,  “Single  Type”  vs  “Distraction  Type”: 


Table  4-14 


A  Comparison  of  ART  for  “Single  Type"  vs  “Distraction  Type”  Approach  Blunders, 
2.4-s  Sensor  Update  Interval,  30-deg  Deviation  Angle,  Far  Range, 

Calm  Flight  Path  Condition 
(3,400-ft  Runway  Separation) 


Blunder  Type 

“Slnule” 

“Distraction” 

#  of  Responses 

49 

49 

Maximum  ART 

7.4 

7.2 

Minimum  ART 

-1.8 

-0.7 

sd 

1.4 

1.3 

Mean  ART 

2.4 

2.4 

Results  indicate  that  t  =  -.13,  P  =  .895  {df  =  48).  Since  the  probability  is  >.001,  the  null 
hypothesis  is  accepted:  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
blunder  type. 

In  summary,  when  comparing  “Distraction  Type”  vs  “Single  Type”  approach  blunders,  no 
significant  difference  in  mean  ART  was  found.  Although  the  difference  in  mean  ART  was  not  found 
statistically  significant  the  reaction  times  indicated  a  general  trend  toward  being  similar  or  a  bit  faster 
in  response  to  the  “Distraction  Type”  vs  “Single  Type.” 

Originally,  these  distraction  blunders  were  designed  to  assess  whether  distractions  (deviations 
by  the  aircraft  soon-to-be  endangered)  would  cause  controllers  to  divert  attention  from  the 
soon-to-be  blundering  aircraft  and  result  in  a  delayed  response  in  breaking  out  the  endangered 
aircraft  Instead,  these  scenarios  sometimes  had  the  opposite  effect  These  sudden  and  unexpected 
deviations  apparently  heightened  controller  awareness  and  resulted  in  faster  breakouts  of  the 
endangered  aircraft.  It  was  determined  that  these  distractions  were  quite  artificial  and  not  to  be 
expected  in  normal  operations.  Therefore,  as  mentioned  in  Section  2.2.1  regarding  Study  II,  the 
distractions  were  removed  and  those  blunders  were  converted  to  "Single  Type”  approach  blunders. 

4. 1.6.1  Effect  of  Sensor  Update  Interval  on  “Fast/Slow  Type”  Approach 
Blunders 

The  results  for  blunder  type  showed  that  in  “Single  Type”  approach  blunders,  whether  the 
sensor  update  interval  was  1.0  s  or  2.4  s,  the  ART  remained  basically  the  same.  However,  the  results 
also  showed  that  in  “Fast/Slow  Type”  approach  blunders  presented  at  1.0-s  sensor  update  interval, 
the  ART  tended  to  be  slower  than  the  ART  obtained  in  the  2.4-s  sensor  update  interval  blunders.  The 
pertinent  information  from  Tables  4-7  through  4-10  is  summarized  below  in  Table  4-15. 
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Table  4-15 


Mean  ART  for  “Fast/Slow  Type”  Approach  Blunders,  15-deg  Deviation  Angle, 
Near  and  Far  Range,  Calm  Flight  Path  Condition, 

Presented  by  Range  and  Sensor  Update  Interval 
(3,400-ft  Runway  Separation) 


Sensor  Update  Interval 

1.0  s 

2.4  s  l 

Near  Range  (Mean  ART) 

2.9 

2.0 

Far  Range  (Mean  ART) 

4.9 

3.8 

To  assess  the  effect  of  sensor  update  interval  in  the  “Fast/Slow  Type”  approach  blunders,  two 
paired-sample  t-tests  were  performed,  one  for  the  approach  blunders  which  occurred  at  near  range 
and  one  for  the  approach  blunders  which  occurred  at  far  range. 

(1)  “Fast/Slow  Type”  approach  blunder,  15-deg  deviation  angle,  near  range,  calm  flight 
path  condition,  1.0-s  vs  2.4-s  sensor  update  interval: 

Results  indicate  that  t  -  2.30,  P  -  .026  {df  =  46).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
sensor  update  interval. 

(2)  “Fast/Slow  Type”  approach  blunder,  15-deg  deviation  angle,  far  range,  calm  flight 
path  condition,  1.0-s  vs  2.4-s  sensor  update  interval: 

Results  indicate  that  t  =  1.94,  P  -  .058  {df  =  49).  Since  the  probability  is  >.001,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
sensor  update  interval.  However,  it  should  be  noted  how  close  the  probability  came  to 
reaching  the  level  of  significance,  indicating  a  trend  toward  a  significant  difference  due 
to  sensor  update  interval. 

In  summary,  there  was  no  significant  difference  in  the  mean  ARTs  of  “Fast/Slow  Type” 
approach  blunders  that  could  be  attributed  to  sensor  update  interval. 

4.1.7  Effect  of  Controller  Experience 

The  novice  group,  while  radar  qualified,  had  no  Monitor  Controller  experience  prior  to  the 
simulation.  In  the  experienced  group,  the  mean  level  of  experience  as  a  Monitor  Controller  was 
approximately  5  years.  The  range  of  experience  was  generally  2  to  5  years.  However,  three 
controllers  had  longer  experience  of  7,  11,  and  12  years.  In  Table  4-16,  it  is  seen  that  the  means 
indicate  that  the  experienced  Monitor  Controllers  responded  approximately  0.9  s  later  than  the 
controllers  who  had  no  previous  monitoring  experience. 
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Table  4-16 


A  Comparison  of  Overall  Alert  Response  Times  of 
Experienced  vs  Novice  Monitor  Controllers 


Monitor  Controller 
Experience 

Novice 

Experienced 

Number  of  Controllers 

24 

26 

Number  of  Responses 

572 

614 

Standard  Deviation  (s) 

1.9 

2.2 

Mean  (s) 

2.4 

3.3 

A  t-test  for  independent  samples  was  performed  in  order  to  assess  the  difference  in  mean 
ART,  novice  vs  experienced  controller.  Results  indicate  that  t  =  -7.07,  P  =  .000  (separate  variance 
estimate  procedure,  df  -  1181.71).  This  was  a  between-subject  comparison,  and  the  probability 
criteria  set  to  indicate  significant  difference  is  .05.  Since  the  probability  is  <.05,  the  null  hypothesis  is 
rejected;  i.e.,  there  is  a  significant  difference  in  mean  ART  due  to  controller  experience  level. 

In  summary,  this  effect  of  controller  experience  level  is  very  evident  when  examining 
Figure  4-2,  which  shows  the  mean  ART  for  experienced  vs  novice  controllers  for  each  of  24 
approach  blunders.  For  all  but  one  of  the  approach  blunders,  the  experienced  Monitor  Controllers 
were  slower  than  the  novice  Monitor  Controllers.  These  phenomena  occurred  in  response  to 
simulations  regardless  of  whether  the  sensor  update  interval  was  1.0  s  or  2.4  s. 
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One  might  expect  the  experienced  Monitor  Controllers  to  have  reacted  more  quickly. 
However,  it  may  be  that  the  more  experience  one  has  in  a  situation,  the  higher  the  awareness  of  the 
many  things  to  consider.  It  also  may  be  that  with  experience,  the  controller  has  the  confidence  to  let 
the  situation  develop  further  before  it  is  necessary  to  intervene.  In  addition,  the  novice  group  may 
have  had  more  of  a  quick-trigger  response  of  seeing  and  hearing  the  alert,  and  then  reacting. 

4.1.8  Another  Measure  of  Effectiveness  -  The  Rate  of  False  Breakouts 

The  simulation  presented  opportunities  for  controllers  to  break  out  aircraft  which  were  not 
scripted  to  be  involved  in  a  blunder.  These  opportunities  were  of  two  distinct  types,  the  deliberate 
and  inadvertent  opportunities. 

The  deliberate  opportunities  included  tracks  deviating  toward  the  NTZ  at  an  angle  of  5  to 
20  deg,  tracks  gradually  drifting  off  course  toward  the  NTZ,  and  tracks  which  had  drifted  away  from 
the  NTZ,  gradually  coming  back  toward  the  localizer  course  and  then  overshooting  it.  The 
inadvertent  opportunities  arose  from  normal  tracks,  generated  from  the  Memphis  TNSE  data  during 
turbulent  conditions.  In  these  cases,  a  combination  of  radar  noise  and  increased  flight  technical  error 
created  flight  paths  that  activated  the  Caution  Alert  The  inadvertent  opportunities  more  closely 
modeled  the  situation  that  might  occur  as  a  result  of  normal  pilot  technique.  The  deliberate 
opportunities  represented  more  erratic  piloting.  The  only  difference  between  one  of  these  and  an 
approach  blunder  is  whether  the  aircraft  enters  the  NTZ  before  turning  back  to  the  approach  course. 

Table  4-17  presents  the  data  on  unnecessary/false  breakouts.  The  results  are  categorized  by 
sensor  update  interval  and  deliberate  or  inadvertent  opportunity.  It  should  be  emphasized  that  the 
percentages  shown  are  false  breakouts  as  a  percentage  of  breakout  opportunities,  not  percentages  of 
total  flights,  which  would  of  course  be  significantly  smaller. 

These  results  were  included  in  the  Precision  Runway  Monitor  Demonstration  Report  [5].  The 
data  were  derived  from  the  responses  of  20  of  the  25  pairs  of  controllers.  (Preliminary  analysis  of 
the  remaining  5  pairs  of  controllers  did  not  significantly  change  the  fmdings.  Further  analysis  was 
not  warranted  and,  therefore,  not  performed.) 

Table  4-17 

False  Breakouts 
(3,400-ft  Runway  Separation) 


Update  Interval 

1.0  s 

r  2.4  S  1 

Type  of  Breakout  Opportunity 

Deliberate 

Inadvertent 

Deliberate 

Inadvertent 

Number  of  Pairs  of  Controllers 

20 

20 

20 

20 

Total  Nuisance  Caution  Alerts 

160 

1380 

90 

1170 

Total  False  Breakouts 

14 

1 

11 

4 

Percentage  of  False  Breakouts 

8.75% 

<.1% 

12% 

<.4% 

False  Breakout  Rate* 

87 

<1 

120 

<4 

*per  1 ,000  Falsa  Breakout  Opportunities 

Table  4-17  shows  that  the  false  breakout  rate  was  less  when  the  faster  sensor  update  interval 
was  used.  This  was  found  for  both  types  of  breakout  opportunities. 
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False  breakouts  occurred  primarily  from  the  so-called  deliberate  opportunities.  Each  of  these 
involved  aircraft  approaching  to  within  100  or  200  ft  of  the  NTZ,  or  deviating  left  and  right  off 
course  several  times.  The  aircraft  would  eventually  return  to  course,  but  the  controller  had  no  way  of 
anticipating  this.  By  breaking  out  either  the  apparent  blunderer  or  the  adjacent  aircraft,  the  controller 
acted  on  a  judgment  that  NTZ  penetration  was  imminent. 

Examination  of  actual  approach  data  collected  at  Memphis  indicates  that  flight  paths  similar 
to  the  deliberate  breakout  opportunities  are  rare,  occurring  in  less  than  1%  of  the  recorded 
approaches.  Therefore,  the  false  breakout  rate  per  approach,  of  the  highest  number  in  the  tables 
(12%),  reduces  to  about  1  per  1,000. 

The  inadvertent  opportunities  were  believed  more  typical  of  what  would  be  expected  to  be  an 
actual  false  breakout  opportunity.  The  low  inadvertent  rate  suggests  that  controllers  are  able  to 
tolerate  a  high  number  of  nuisance  Caution  Alerts  in  turbulent  conditions.  Because  all  aircraft 
evidence  increased  difficulties  in  tracking  the  localizer  compared  with  calm  wind  conditions, 
controllers  expected  some  deviations,  and  the  occasional  Caution  Alerts  did  not  prompt  them  to  break 
aircraft  out  unnecessarily. 

In  summary,  whether  the  sensor  update  interval  was  1.0  s  or  2.4  s,  controllers  had  a  low  rate 
of  false  breakouts  in  response  to  simulations  with  typical  flight  path  variations.  In  both  the  deliberate 
and  inadvertent  opportunities,  the  faster  sensor  update  interval  resulted  in  less  false  breakouts. 

4.2  RESPONSES  TO  SIMULATIONS  OF  APPROACH  BLUNDERS  PRESENTED  AT  4.8-S 
SENSOR  UPDATE  INTERVAL,  3,400-FT  RUNWAY  SEPARATION 

4.2.1  Effect  of  4.8-s  Sensor  Update  Interval 

In  order  to  assess  the  effect  of  the  4.8-s  sensor  update  interval  when  runway  separation  was 
3,400  ft,  the  responses  of  the  controllers  to  the  “Single  Type”  approach  blunders  presented  at  that 
sensor  update  interval  were  studied.  These  data  were  collected  during  Weeks  1  through  12  of  testing, 
i.e.,  involving  24  controllers.  The  data  were  collected  to  assess  the  benefit  of  the  PRM  system  in  the 
following  hypothetical  situation:  the  use  of  new  displays  and  one-milliradian  radar  accuracy,  3,400  ft 
runway  separation,  without  changing  the  sensor  update  interval  from  that  of  the  production  Mode  S 
sensor,  i.e.,  use  of  4.8-s  sensor  update  interval. 

In  Table  4-18,  the  ARTs’  results  are  presented  for  the  three  sensor  update  intervals.  These 
results  are  from  the  responses  to  the  six  “Single  Type”  approach  blunders  in  common  in  all  three 
sensor  update  interval  conditions. 
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Table  4-18 


A  Comparison  of  ART  for  “Single  Type"  Approach  Blunders  Presented  at  Three  Sensor 

Update  Interval  Conditions 
(3,400-ft  Runway  Separation) 


Update  Interval 

1.0  8 

2.4  s 

4.8  S 

#  of  Responses 

121 

120 

121 

Maximum  ART 

9.8 

11.5 

12.24 

Minimum  ART 

-2.6 

-2.6 

-0.64 

sd 

2.3 

2.3 

2.3 

Mean  ART 

3.3 

3.4 

4.2 

Two  paired-sample  t-tests  were  performed  to  assess  the  effect  of  the  slower  sensor  update 
interval  of  4.8  s  vs  1.0-s  and  2.4-s  sensor  update  intervals: 

(1)  1.0-s  vs  4.8-s  sensor  update  interval: 

Results  indicate  that  r=  -3.45,  P  =  .001  (df  =  120).  Since  the  probability  is  .001,  the 
null  hypothesis  is  rejected;  i.e.,  there  is  a  significant  difference  in  mean  ART  due  to 
sensor  update  interval.  This  is  a  very  strong  effect.  The  probability  of  this  effect 
occurring  by  chance  due  to  sampling  error  is  approximately  1  in  1,000. 

(2)  2.4-s  vs  4.8-s  sensor  update  interval: 

Results  indicate  that  t  =  -2.86,  P  =  .005  (df  =  119).  Since  the  probability  is  >.001  the 
null  hypothesis  is  accepted,  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
sensor  update  interval.  However,  although  this  probability  value  did  not  meet  the 
stringent  criteria  set  through  application  of  the  Bonferroni  Adjustment,  it  indicates  a 
strong  trend  toward  significance. 

In  summary,  results  indicate  approximately  1.0-s  difference  in  mean  ART  in  the  4.8-s  sensor 
update  condition  vs  the  mean  ART  in  the  other  two  sensor  update  intervals.  Controllers  were  1.0  s 
slower  in  responding  when  the  4.8-s  sensor  update  interval  was  used.  So,  despite  the  new  displays  and 
greater  sensor  accuracy,  use  of  the  4.8-s  sensor  update  interval  resulted  in  delayed  response  times. 

As  a  result  of  this  finding,  the  decision  was  made  to  study  the  use  of  4.8-s  sensor  update 
interval,  with  the  new  displays  and  greater  sensor  accuracy,  but  to  change  the  runway  separation  to 
4,300  ft.  This  testing  was  conducted  during  Weeks  13  through  25,  i.e.,  involving  26  controllers  and  is 
discussed  in  Section  4.4. 

Before  moving  on  to  those  findings,  we  will  consider  the  effect  of  deviation  angle  in  approach 
blunders  with  3,400-ft  runway  separation,  4.8-s  sensor  update  interval.  This  was  done  in  order  to  see 
if  the  findings  would  be  similar  to  the  findings  regarding  the  other  two  sensor  update  intervals,  i.e., 
that  controllers  responded  approximately  1.0  s  more  rapidly  to  approach  blunders  with  30-deg  vs 
15-deg  deviation  angle  (see  Section  4.1.2). 

4.2.2  Effect  of  Deviation  Angle 

In  order  to  assess  the  effect  of  deviation  angle  in  approach  blunders  with  3,400  ft  runway 
separation,  4.8-s  sensor  update  interval,  “Single  Type”  approach  blunders  were  studied.  Table  4-19 
presents  the  ARTs’  results. 
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Table  4-19 


A  Comparison  of  ART  for  “Single  Type”  Approach  Blunders, 
4.8-s  Sensor  Update  Interval,  15-deg  vs  30-deg  Deviation  Angle 
(3,400-ft  Runway  Separation) 


Deviation  Angle 

15  deg 

30  deg 

#  of  Responses 

45 

45 

Maximum  ART 

9.6 

8.0 

Minimum  ART 

0.7 

-0.6 

sd 

2.4 

2.0 

Mean  ART 

4.0 

3.7 

A  paired-sample  t-test  was  performed  to  assess  the  effect  of  deviation  angle.  Results  indicate 
that  t  =  0.6,  P  -  .508  (df  =  44).  Since  the  probability  is  >.001,  the  null  hypothesis  is  accepted,  i.e., 
there  is  no  significant  difference  in  mean  ART  due  to  deviation  angle. 

In  summary,  while  controller  response  to  approach  blunders  with  1.0-s  and  2.4-s  sensor 
update  intervals  showed  a  significant  difference  in  mean  ART  attributable  to  deviation  angle,  this 
effect  was  not  found  in  the  4.8-s  sensor  update  interval  condition. 

This  is  probably  due  to  the  fact  that  whether  a  deviation  is  more  or  less  severe,  the  controller 
decided  he/she  could  not  wait  for  an  additional  sensor  update  before  making  a  breakout  decision. 
Waiting  another  4.8  s  was  not  acceptable  to  the  controllers. 

4.3  EFFECT  OF  RUNWAY  SEPARATION  (3,400  FT  VS  4,300  FT)  ON  RESPONSES  TO 
SIMULATIONS  OF  APPROACH  BLUNDERS  PRESENTED  AT  4.8-JS  SENSOR  UPDATE 
INTERVAL 

In  order  to  assess  the  effect  of  the  4.8-s  sensor  update  interval  when  runway  separation  was 
3,400  ft  vs  4,300  ft,  the  responses  of  the  controllers  to  the  “Single  Type”  approach  blunders 
presented  at  that  sensor  update  interval  were  studied.  The  data  collected  during  Weeks  1  through  12 
of  testing  with  3,400  ft  runway  separation  were  compared  with  the  data  collected  during  Weeks  13 
through  25  of  testing  with  4,300  ft  runway  separation.  This  was  done  in  order  to  assess  the  viability 
of  using  the  PRM  system  while  keeping  the  current  standard  of  4,300-ft  runway  separation. 

In  Table  4-20,  the  ARTs’  results  are  presented  for  the  “Single  Type”  approach  blunders 
presented  at  4.8-s  sensor  update  interval,  3,400  vs  4,300  ft. 

Table  4-20 

A  Comparison  of  ART  for  “Single  Type”  Approach  Blunders 
Presented  at  4.8-s  Sensor  Update  Intervals 
(3,400-ft  vs  4,300-ft  Runway  Separation) 


Runway  Separation 

3,400  ft 

4,300  ft 

#  of  Responses 

121 

143 

sd 

2.3 

3.1 

Mean  ART 

4.2 

1.6 
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A  t-test  for  independent  samples  was  performed  to  assess  the  significance  of  the  difference 
between  the  two  mean  ARTs.  (A  paired-sample  t-test,  could  not  be  used  for  this  analysis.  In  this 
case,  there  was  not  a  paired  sample,  i.e.,  the  comparison  of  an  individual's  performance  in  one 
condition  vs  the  same  individual's  performance  in  another  condition.  The  comparison  was  made 
between  one  group  of  controllers  who  participated  in  the  first  half  of  the  study  and  another  group  of 
controllers  who  participated  in  the  second  half  of  the  study.) 

Using  a  separate  variance  estimate,  2-tail  probability,  results  indicate  that  t  =  8.58,  P  =  .000 
C df  -  257.25).  This  was  a  between-subject  comparison  and  the  probability  criteria  set  to  indicates 
significant  difference  is  .05.  Since  the  probability  is  <.05,  the  null  hypothesis  is  rejected,  i.e.,  there  is 
a  significant  difference  in  mean  ART  due  to  runway  separation.  This  is  a  very  strong  effect  The 
probability  of  this  effect  occurring  by  chance  due  to  sampling  error  is  less  than  1  in  1,000. 

Controller  ART  was  significantly  faster  when  the  runway  separation  was  increased  from 
3,400  ft  to  4,300  ft  One  possible  explanation  for  the  extreme  difference  in  mean  ART  concerns  the 
presence  of  a  larger  NOZ  in  the  4,300-ft  runway  separation  condition.  The  controllers  had  the 
opportunity  to  observe  the  blundering  aircraft  for  one  or  two  more  radar  intervals  prior  to  NTZ 
penetration.  When  the  Caution  Alert  sounded,  the  aircraft  was  far  from  the  centerline.  There  was 
little  likelihood  that  this  was  a  TNSE  deviation  and,  therefore,  the  controller  was  confident  in  making 
the  breakout  decision  and  responded  quickly. 

This  explanation  is  supported  in  the  data  on  unnecessary  breakouts.  Table  4-21  presents  data 
on  unnecessary  breakouts  at  the  4.8-s  sensor  update  interval  and  compares  3,400-ft  vs  4,300-ft 
runway  separation.  No  nuisance  Caution  Alerts  occurred  at  4.8  s  and  4,300  ft  and,  therefore,  no 
unnecessary  breakouts  occurred,  even  for  the  deliberate  opportunities  (“deliberate”  and 
“inadvertent”  opportunities  were  defined  in  Section  4.1.8). 

Table  4-21 

Unnecessary  Breakouts  -  Comparison  of  Runway  Separations  at 
4.8-s  Sensor  Update  Interval 


Runway  Separation 

3,400  ft 

4,300  ft  ””~1 

Type  of  Breakout  Opportunity 

Deliberate 

Inadvertent 

Deliberate 

Inadvertent 

Number  of  Pairs  of  Controllers 

14 

14 

12 

12 

Total  Nuisance  Caution  Alerts 

448 

996 

0 

0 

Total  False  Breakouts 

5 

17 

0 

0 

Percentage  of  False  Breakouts 

1.1% 

1.75% 

0 

0 

False  Breakout  Rate* 

11 

17 

0 

0 

•par  1 ,000  False  Breakout  Opportunities 


Since  with  4,300-ft  runway  separation,  no  nuisance  Caution  Alerts  occurred,  the  controllers 
learned  that  when  they  did  receive  a  Caution  Alert  it  was  probably  an  approach  blunder  in  progress. 
It  is,  therefore,  reasonable  to  expect  that  controllers  would  have  greater  confidence  in  making  the 
breakout  decisions  and,  therefore,  they  reacted  more  quickly  when  the  runway  separation  was 
4,300  ft. 
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4.4  RESPONSES  TO  SIMULATIONS  OF  APPROACH  BLUNDERS  PRESENTED  AT  4.8-S 
SENSOR  UPDATE  INTERVAL,  4,300-FT  RUNWAY  SEPARATION 

4.4.1  Effect  of  Deviation  Angle 

In  order  to  assess  the  effect  of  deviation  angle  in  approach  blunders  with  4,300-ft  runway 
separation.  4.8-s  sensor  update  interval,  “Single  Type”  approach  blunders  were  studied.  Table  4-22 
presents  the  ARTs  results  for  the  “Single  Type”  approach  blunders  in  common  in  the  15-deg  and 
30-deg  conditions. 


Table  4-22 

A  Comparison  of  ART  for  “Single  Type”  Approach  Blunders, 
15-deg  vs  30-deg  Deviation  Angle 
(4,300-ft  Runway  Separation) 


Deviation  Angle 

15  deg 

30  deg 

#  of  Responses 

47 

47 

Maximum  ART 

8.7 

8.8  ! 

Minimum  ART 

-8.0 

-4.2 

sd 

3.5 

3.0 

Mean  ART 

0.0 

1.0 

A  paired-sample  t-test  was  performed  to  assess  the  effect  of  deviation  angle.  Results  indicate 
that  t  =  -1.58,  P  =  .121  (df=  46).  Since  the  probability  is  >.001,  the  null  hypothesis  is  accepted,  i.e., 
there  is  no  significant  difference  in  mean  ART  due  to  deviation  angle. 

In  summary,  whether  the  deviation  angle  was  15  deg  or  30  deg,  mean  ART  did  not  differ 
significantly.  The  mean  ART  was  1.0  s  or  less.  This  means  that,  on  average,  controllers  responded  at 
the  time  of  the  Caution  Alert  or  within  1.0  s  after  the  Caution  Alert  The  difference  between  0  and 
1.0  s  was  not  found  to  be  statistically  significant 

This  quick  response  time  is  probably  due  to  the  controllers’  high  confidence  in  their 
breakout  decisions.  This  increased  confidence,  seen  in  the  case  of  4,300-ft  runway  separation,  was 
discussed  in  Section  4.3. 

4.4.2  Effect  of  Approach  Blunder  Range 

“Single  Type”  approach  blunders  were  studied  in  order  to  assess  the  effect  of  the  range  at 
which  the  approach  blunder  occurred,  4.8-s  sensor  update  interval,  "Single  Type”  approach 
blunders  were  studied.  Table  4-23  presents  the  ARTs  results.  Within  the  “Single  Type”  approach 
blunders,  blunders  were  presented  at  both  the  near  and  far  range,  30-deg  deviation  angle.  In  the 
15-deg  deviation  angle  condition,  only  near  range  deviations  were  included.  Therefore,  only  results 
from  the  30-deg  deviation  condition  are  presented  below. 
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Table  4-23 


A  Comparison  of  ART  for  “Single  Type”  Approach  Blunders, 
Near  vs  Far  Deviation  Range,  30-deg  Deviation  Angle 
(4,300-ft  Runway  Separation) 


Deviation  Range 

Near 

Far 

#  of  Responses 

22 

22 

Maximum  ART 

8.0 

12.2 

Minimum  ART 

-0.6 

1.2 

sd 

2.0 

2.2 

Mean  ART 

3.1 

5.7 

A  paired-sample  t-test  was  performed  to  assess  the  effect  of  range  in  30-deg  deviation  angle 
approach  blunders.  Results  indicate  that  t  =  -4.77,  P  =  .000  (df  =  21).  Since  the  probability  is  <.001, 
the  null  hypothesis  is  rejected,  i.e.,  there  is  a  significant  difference  in  mean  ART  due  to  range.  This  is 
a  very  strong  effect.  The  probability  of  this  effect  occurring  by  chance  due  to  sampling  error  is  less 
than  1  in  1,000. 

In  summary,  whether  an  approach  blunder  was  presented  at  1.0-s  or  2.4-s  sensor  update 
interval,  when  the  deviation  angle  was  30  deg,  the  ART  was  significantly  slower  for  approach  blunders 
at  far  range  than  near  range.  Mean  ART  was  approximately  2.6  s  slower. 

This  finding  of  slower  ART  at  far  range  is  consistent  with  the  findings  reported  in  Section 
4.1.4  regarding  far  range  and  3,400-ft  runway  separation.  As  discussed  in  that  section,  it  may  be  that, 
at  the  far  range,  the  aircraft  are  not  stable  on  the  localizer  and  TNSE  is  greater.  The  controllers 
therefore  use/allow  more  time  to  watch  the  aircraft  before  making  a  breakout  decision. 

4.4.3  Effect  of  Flight  Path  Condition 

In  order  to  assess  the  effect  of  flight  path  condition  in  approach  blunders  with  4,300-ft 
runway  separation,  4.8-s  sensor  update  interval,  “Single  Type”  approach  blunders,  with  15-deg 
deviation  angle,  and  occurring  at  near  range  were  studied.  Table  4-24  presents  the  ARTs  results  for 
the  “Single  Type”  approach  blunders  in  common  in  the  calm  and  turbulent  flight  path  conditions. 

Table  4-24 

A  Comparison  of  ART  for  “Single  Type”  Approach  Blunders, 

Calm  vs  Turbulent  Flight  Path  Condition,  4.8-s  Sensor  Update  Interval 
(4,300-ft  Runway  Separation) 


Flight  Path  Condition 

Calm 

Turbulent 

#  of  Responses 

23 

23 

Maximum  ART 

5.0 

8.7 

Minimum  ART 

-5.0 

-8.0 

sd 

3.5 

3.6 

Mean  ART 

0.0 

0.0 
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A  paired-sample  t-test  was  performed  to  assess  the  effect  of  flight  path  condition.  Results 
indicate  that  t  =  0.13,  P  -  .895  {df  =  22).  Since  the  probability  is  >.001,  the  null  hypothesis  is 
accepted,  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to  flight  path  condition. 

4.4.4  Effect  of  Approach  Blunder  Type 

In  order  to  assess  the  effect  of  approach  blunder  type  with  4,300  ft  runway  separation,  4.8-s 
sensor  update  interval,  “Single  Type”  approach  blunders  were  compared  to  “Fast/Slow  Type” 
approach  blunders.  Table  4-25  presents  the  ARTs  results  for  “Single  Type”  vs  “Fast/Slow  Type” 
approach  blunders  in  common  in  both  types  of  approach  blunder  type. 

Table  4*25 

A  Comparison  of  ART  for  “Single  Type”  vs  “Fast/Slow  Type”  Approach  Blunders, 
15-deg  Deviation  Angle,  Near  Range,  Calm  Right  Path  Condition, 

4.8-s  Sensor  Update  Interval 
(4,300-ft  Runway  Separation) 


Blunder  Type 

“Single” 

“Fast/Slow” 

#  of  Responses 

24 

24 

Maximum  ART 

5.0 

7.0 

Minimum  ART 

-5.0 

-4.8 

sd 

3.4 

3.7 

Mean  ART 

0.1 

0.0 

A  paired-sample  t-test  was  performed  to  assess  the  effect  of  “Single  Type”  vs  “Fast/Slow 
Type”  approach  blunders.  The  analysis  included  ARTs  to  approach  blunders  with  15-deg  deviation 
angle  and  near  range,  since  a  comparable  data  set  was  present  in  both  the  “Single  Type”  and 
“Fast/Slow  Type”  data  category.  Results  indicate  that  t=  0.16,  P  =  .875  {df  =  23).  Since  the 
probability  is  >.001,  the  null  hypothesis  is  accepted,  i.e.,  there  is  no  significant  difference  in  mean 
ART  due  to  approach  blunder  type,  “Single  Type”  vs  “Fast/Slow  Type.” 

4.5  CONTROLLER  OPINION  ON  THE  USE  OF  THE  PRM  WHEN  RUNWAY 
SEPARATION  IS  3,400  FT 

Controller  survey  forms  were  used  to  obtain  the  opinions  of  the  controllers  on  the 
effectiveness  of  the  PRM  system  and  its  overall  acceptability  for  use.  The  controllers  expressed 
overall  approval  with  the  PRM  system.  Controllers  made  some  recommendations  regarding  personal 
preferences  in  the  manner  in  which  information  was  presented  on  the  display. 

Table  4-26  lists  the  percentage  of  controllers  who  agreed,  disagreed,  or  were  undecided 
regarding  each  survey  statement.  The  complete  text  of  each  survey  statement  is  also  presented, 
accompanied  by  a  summary  of  results  and  any  narrative  comments  made  by  the  controllers. 
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Table  4*26 


roach 


Study  I 

Summary  of  Controller  Survey  Results 


Survey  Item 


2.0  GENERAL  ACCEPTANCE 

2 . 1  Monitor  final  better  than  ARTS 

2.2  High  resolution  color  display  better  for  monitor 
function  than  ARTS  display 

2.3  Automated  alerts  made  r  easier  to  detect  and 
resolve  blunders 

2.4  Approaches  with  runways  separated  by  3,400/ 
3.500  can  be  safely  conducted 


3.0  MONITOR  CONTROLLER  FUNCTIONS 

3 . 1  PRM  is  useful  to  prevent  NTZ  penetration 

3.2  PRM  is  useful  in  resolving  blunders 

3.3  PRM  is  useful  in  detecting  deviations 

3.4  PRM  is  useful  in  monitorina  the  missed 


4.0  NTZ  ALERTS 

4.1  Yellow  Caution  Alert  is  useful 

4.2  Voice  alert  is  useful  (Memphis  only) 

4.3  Red  Wamina  Alert  is  useful 


5.0  DISPLAY  INFORMATION  CONTENT  & 
PRESENTATION 

5 . 1  Information  on  display  is  well  placed  and  useful 

5.2  Written  information  on  display  is  easily  read 

5.3  Color  is  better  than  monochrome 

5.4  deleted 

5.5  Parallel  200-ft  lines  are  useful 

5.6  Color  selection  of  features  is  suitable 


6.0  FEATURES 

6.1  History  Trail 

6.2  Projected  Position  Vector 


7.0  TRAINING 

7.1  T raining  time  was  adequate 

7.2  All  information  was  provided 


8.0  SIMULATION 

8 . 1  Simulated  traffic  density  was  realistic 

8.2  Simulated  blunder  trajectories  were  realistic 

8.3  Simulated  missed  approach  trajectories  were 
realistic 

8.4  Audio  portion  of  simulation  was  realistic 


Survey  Section  2:  General  Acceptance 

PRM  enabled  me  to  monitor  the  final  approach  better  than  the  existing 
Automated  Radar  Terminal  System  (ARTS)  system. 

PRM's  high  resolution  color  monitor  is  better  for  the  monitor  function  than 
the  current  ARTS  system. 
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Statement  2.3  The  PRM  display  with  the  automated  alerts  made  it  easier  to  detect  and 
resolve  potential  and  actual  blunders/deviations  better  than  the  existing 
ARTS  system. 

Fifty  controllers  unanimously  agreed  with  the  above  three  statements.  Controllers  made 
comments  indicating  a  high  level  of  acceptability  of  the  system.  The  comments  of  many  controllers 
were  similar  to  this  comment  made  by  one  controller,  “The  PRM  system  is  very  impressive,  and  the 
system  should  be  implemented  as  soon  as  possible  to  airports  that  need  to  relieve  congestion  and 
controller  workload  and,  most  importantly,  to  enhance  safety.”  When  comparing  the  PRM  system  to 
the  current  ARTS  system,  controllers  described  PRM  as  being:  “a  vast  improvement,”  commenting 
that  there  is  “no  comparison"  and  that  “this  system  is  much  more  accurate.” 

Controller  comments  on  the  automated  alerts  indicated  unanimous  approval.  Comments 
indicated  that  the  alerts  are  invaluable  when  considering  ambient  noise  and  distractions.  One 
controller  commented  that  the  alerts  are  an  “Excellent  idea,  considering  the  boredom  factor  at  final 
monitor  is  very  high,  and  your  attention  drifts."  Another  controller  commented,  “Visual  and 
audible  alarms  are  an  absolute  must  for  forewarning  controllers.” 

Statement  2.4  Independent  IFR  approaches  to  runways  separated  by  3,400/3,500  ft  can  be 
safely  conducted  using  the  PRM. 

Forty-eight  controllers  agreed  with  this  statement  The  two  controllers  who  were  undecided 
indicated  that  before  making  a  decision,  they  would  have  liked  more  time  in  which  to  become 
familiar  with  the  system. 


Survey  Section  3:  Monitor  Controller  Functions 

Statement  3.1  PRM  is  useful  as  a  final  approach  monitor  to  prevent  penetration  of  the 
NTZ. 

Forty-nine  controllers  agreed  with  this  statement.  Many  controllers  stated  that  the 
combination  of  the  faster  sensor  update  interval  and  the  presence  of  Warning  Alerts  greatly  improved 
the  safety  of  this  type  of  operation.  The  one  controller  who  disagreed  stated,  “Nothing  will  prevent 
penetration  of  the  zone,  if  the  aircraft  angle  of  deflection  is  large  leaving  his  final.” 

Statement  3.2  PRM  is  useful  in  resolving  approach  blunders  once  they  have  occurred. 

Forty-eight  controllers  agreed  with  this  statement.  Comments  from  one  controller  who 
disagreed  emphasized  the  need  for  new  procedures  to  be  developed  for  conducting  simultaneous 
parallel  approaches. 

Statement  3.3  PRM  is  useful  in  detecting  deviations  from  the  designated  approach  course. 

Fifty  controllers  unanimously  agreed  that  PRM  is  useful  in  detecting  deviations  from  the 
designated  approach  course.  Controllers  commented  on  the  benefits  of  increased  magnification. 
One  controller's  comment  summarizes  what  many  of  the  controllers  expressed.  He  referred  to  the 
increased  magnification,  saying,  "this  makes  small  deviations  more  readily  detectable.” 
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Statement  3.4  PRM  is  useful  in  monitoring  simultaneous  missed  approach  to  ensure  that 
the  required  divergence  is  achieved. 

Forty-four  controllers  agreed  with  this  statement.  Based  on  the  comments  from  controllers 
who  either  disagreed  or  were  undecided,  they  were  not  saying  that  PRM  is  not  useful  in  monitoring 
the  missed  approach.  They  commented  that  monitoring  the  missed  approach  should  be  the 
responsibility  of  the  local  controller  and  not  the  Monitor  Controller. 

Survey  Section  4:  NTZ  Alerts 

Statement  4.1  The  Yellow/Caution  Visual  Alert,  predicting  “x”  seconds  or  less  until  NTZ 
penetration,  is  useful. 

Forty-nine  controllers  agreed  with  this  statement  The  one  controller  who  disagreed  about  its 
usefulness  reported  having  difficulty  seeing  the  yellow  color  of  a  particular  aircraft  ID  when  he  was 
not  looking  directly  at  it.  No  other  controller  reported  having  this  difficulty. 

Statement  4.2  The  Voice  Alert  accompanying  the  Yellow/Caution  Visual  Alert  is  useful. 

All  but  one  controller  agreed  that  the  voice  alert  was  useful.  The  one  controller  who 
disagreed  reported  preferring  a  “beep”  which  would  be  heard  externally  from  the  headset  audio. 

Statement  4.3  The  Red/Waming  Visual  Alert,  indicating  that  NTZ  penetration  has 
occurred,  is  useful. 

Forty-nine  controllers  agreed  with  this  statement.  One  controller  who  agreed  commented  that 
the  Warning  Visual  Alert  eliminates  questions  regarding  whether  or  not  penetration  of  the  NTZ  has 
occurred. 


Survey  Section  5:  Display  Information  Content  and  Presentation 

Statement  5.1  The  information  presented  in  the  PRM  display  is  well-placed  and  easily 
visible. 

Forty-nine  controllers  agreed  with  this  statement.  The  one  controller  who  was  undecided 
stated  that  he  needed  more  time  with  the  system  before  making  a  judgment 

Statement  5.2  The  written  information  presented  in  the  PRM  display  is  easily  read. 

Forty-eight  controllers  agreed  with  this  statement  Two  controllers  who  disagreed  stated  that 
the  menu  was  “cluttered.”  Their  disagreement  referred  to  the  menu  and  not  the  readability  of  the 
display  text  The  menu  structure  was  not  under  study  in  this  experiment.  The  current  menu  structure 
is  lengthy,  since  it  includes  items  which  are  used  by  the  experimenter  in  setting  up  the  simulation. 
The  actual  menu  that  will  be  seen  by  a  Monitor  Controller  will  be  greatly  streamlined. 

Statement  5.3  The  color  display  is  more  effective  than  a  monochrome  display. 

Fifty  controllers  unanimously  agreed  with  this  statement. 

Statement  5.4  deleted 
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Statement  5.5  The  parallel  200-ft  lines  are  a  useful  aid  in  detecting  deviations  from  the 
approach  course  and  in  predicting  the  potential  for  an  NTZ  penetration. 

Forty-six  controllers  agreed  that  the  lines  are  useful.  One  controller  disagreed  and  three  were 
undecided.  The  majority  of  controllers  stated  that  the  lines  help  to  detect  deviations  at  the  earliest 
time.  Of  the  controllers  who  were  undecided  or  disagreed,  one  stated  that  he/she  did  not  use  the  lines 
and  found  them  to  be  “unnecessary  clutter.”  One  controller  suggested  increasing  the  distance 
between  lines,  thereby  reducing  the  number  of  lines.  Some  controllers  stated  that  the  lines  should  be 
optional,  and  expressed  that  some  controllers  would  benefit  from  their  use  and  some  would  not. 

Statement  5.6  The  color  selection  for  the  features  on  the  display  is  suitable. 

Forty-seven  controllers  agreed  with  this  statement.  Three  controllers  disagreed.  One 
controller  commented  that  the  predictor  lines  should  be  a  color  which  would  “stand  out”  more. 
Another  controller  commented  that  the  “Caution  yellow  should  be  brighter.”  This  was  the  same 
controller,  discussed  above  (Statement  4.1),  who  had  difficulty  seeing  the  yellow  color  of  a  particular 
aircraft  ID  when  he  was  not  looking  directly  at  it.  No  other  controller  reported  having  any  difficulty 
seeing  the  yellow  color.  This  one  controller's  difficulty  perceiving  the  yellow  color  illustrates  the 
value  of  having  redundancy  in  the  alert  system.  In  addition  to  visual,  color-coded  alerts,  there  are 
accompanying  audible  alerts. 


Survey  Section  6:  Features 

Statement  6.1  The  History  Trail  is  useful  in  assisting  you  to  perform  the  Monitor 
Controller  task. 

Thirty-seven  controllers  agreed  that  it  is  useful.  Of  the  controllers  who  disagreed  cr  were 
undecided,  many  stated  that  the  use  of  this  feature  should  be  a  matter  of  personal  preference. 

Statement  6.2  The  projected  Position  Vector  is  useful  in  assisting  you  to  perform  the 
Monitor  Controller  task. 

Forty-seven  controllers  agreed  that  it  is  useful.  Many  controllers  stated  that  this  feature  is  one 
of  the  best  aspects  of  the  system.  One  controller  commented,  “I  would  like  this  to  change  color  if 
the  projected  turn  is  more  than  10  deg.” 

Survey  Section  7:  Training 

Statement  7.1  Adequate  training  time  was  provided  to  become  familiar  with  the  display 
before  testing  began. 

Forty-six  controllers  agreed  with  this  statement.  Two  of  the  four  controllers,  who  were 
undecided  stated  that  they  would  have  liked  a  little  more  time  working  with  the  display  before  testing 
began. 

Statement  1.2  All  information  needed,  to  aid  me  in  performing  the  monitoring  task,  was 
provided. 
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Forty-eight  controllers  agreed  with  this  statement.  Overall,  controllers  commented  that  the 
training  was  “excellent”  and  that  “there  was  always  someone  there  if  a  question  or  concern  arose.” 
One  controller  who  disagreed  said  that  the  ability  to  change  intensities  was  not  fully  explained. 

Survey  Section  8:  The  Simulation 

Statement  8.1  The  simulated  traffic  density  was  realistic. 

Forty-nine  controllers  agreed  with  this  statement.  The  controller  who  disagreed  commented 
that  there  should  have  been  “more  bumps  into  the  NTZ." 

Statement  8.2  The  simulated  aircraft  “blunder”  trajectories  were  realistic. 

Controller  opinion  was  split  on  this  item.  Most  controllers  who  disagreed  gave  one  of  the 
following  reasons: 

1)  Some  controllers  found  it  difficult  to  adjust  to  the  magnification  of  the  x-  and  y-axis. 
The  magnification  makes  the  angle  of  the  deviation  appear  more  severe  than  it  actually 
is.  Some  controllers  commented  that  the  angle  of  deviation  was  too  great  and, 
therefore,  unrealistic.  In  the  simulation,  the  angle  of  the  deviations  did  not  exceed 
30  deg,  but  some  controllers  said  that  it  was  greater. 

2)  Some  controllers  said  that  there  were  too  many  emergencies  and  that  this  was 
unrealistic.  The  simulation  did  intentionally  show  many  more  blunders  than  one  would 
experience  in  actual  operations.  Actual  blunders  are  infrequent  and,  therefore,  difficult 
to  study.  Through  simulation  a  number  and  variety  of  blunders  were  presented  in 
order  to  obtain  valuable  data  on  controller  responses. 

Statement  8.3  The  simulated  aircraft  missed  approach  trajectories  were  realistic. 

lne  majority  of  controllers  agreed  with  this  statement.  Many  controllers  who  disagreed  stated 
that  the  scenarios  which  depicted  two  aircraft  on  adjacent  approach  paths,  making  a  simultaneous 
missed  approach,  and  then  simultaneously  blundering  toward  the  NTZ,  i.e.,  toward  each  other,  was 
highly  unlikely.  Some  controllers  commented  that,  since  the  probability  of  this  event  is  so  small,  this 
event  should  not  have  been  included  in  the  scenarios. 

Statement  8  A  The  audio  portion  of  the  simulation  was  realistic. 

The  majority  of  controllers  agreed  with  this  statement.  For  the  recorded  audio,  one  controller 
and  one  pilot  spoke  the  parts  of  all  pilots  and  all  local  controllers.  One  controller  who  disagreed 
commented  that  the  voices  were  too  monotonous.  A  few  controllers  commented  that  the  background 
audio  was  “too  wordy.” 


Survey  Section  9:  Comments 

Controllers  were  asked  to  comment  on  any  changes  to  the  simulation  which  might  enhance  its 
realism.  Controllers  suggested  simulating  the  actual  work  environment  that  they  experience.  They 
sited  the  presence  of  many  more  distractions  in  a  live  TRACON.  Controllers  commented  that  more 
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speed  changes  should  be  included  in  the  simulation.  Approximately  three  speed  changes  per  hour 
were  scripted  into  the  Memphis  simulation. 

Controllers  were  asked  to  identify  any  factors  in  the  simulation  which  might  have  affected  the 
quality  of  the  reaction  time  measurement.  Some  controllers  commented  that  there  were  many  more 
blunders  than  one  would  encounter  during  actual  monitoring.  One  controller  commented  that  this 
created  stress  and  may  have  slowed  his  responses.  Another  controller  commented  that  this 
heightened  his  anticipation  and  may  have  quickened  his  responses. 

Controllers  were  asked  to  make  any  additional  comments  regarding  the  simulation,  the 
display,  or  the  study  procedures.  Many  positive  comments  were  received.  Controllers  reported  being 
impressed  with  the  system  and  generally  stated  that  it  should  be  implemented  as  soon  as  possible. 

Some  concerns  were  voiced  by  controllers.  The  controllers  preferred  the  1.0-s  and  2.4-s 
sensor  update  intervals  to  the  4.8-s  sensor  update  interval.  The  4.8-s  sensor  update  interval  was  said 
to  be  “too  slow.”  There  was  also  concern  that  problems  may  be  encountered  with  frequency 
congestion,  especially  when  the  4.8-s  sensor  update  interval  is  used.  A  few  controllers  were 
concerned  that  the  controller's  communication  transmission  may  be  blocked  at  a  critical  time  by  an 
aircraft  transmitting  on  the  frequency.  There  were  also  concerns  that  in  using  the  4.8-s  sensor  update 
interval,  many  unnecessary  corrective  headings  may  have  to  be  issued. 
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5.  RESULTS  OF  STUDY  H 


In  Section  5,  the  results  of  Study  II  are  discussed.  This  section  focuses  on  controller  response 
to  simulations  presented  at  1.0-s  and  2.4-s  sensor  update  intervals,  3,000-ft  runway  separation.  Where 
applicable,  results  of  Study  I  (3,400-ft  runway  separation)  are  compared  to  results  of  Study  II. 

Throughout  Section  5,  there  is  a  brief  description  of  each  analysis  performed.  ART  results 
which  are  statistically  significant  are  underlined.  Throughout  Section  5  within-subject  comparisons 
are  made  and  the  probability  criteria  set  for  interpreting  a  significant  difference  between  means  is 
.005  or  less  (computation  of  probability  criteria  is  discussed  in  Section  3.3.1.)  Results  of  the 
Controller  Opinion  Survey  used  for  Study  II  are  presented  in  Section  5.5. 

Before  delving  into  the  details  of  each  analysis  area,  an  “Overview  of  Findings”  is  presented. 
This  gives  the  reader  a  quick  look  at  the  answers  to  the  research  questions  asked  in  Study  II  and  listed 
in  Section  2.2.2.  In  addition,  reference  is  made  to  the  section  in  which  greater  detail  on  findings  can 
be  found. 
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Overview  of  Findings 


Question  1:  How  does  sensor  update  interval  affect  reaction  time? 

Findings:  Whether  the  sensor  update  interval  was  1.0  s  or  2.4  s,  the  ART  did  not  differ 

significantly.  Mean  ART  for  both  sensor  update  intervals  was  approximately  2.5  s. 
_ (For  details,  see  Section  5.1.) _ 

Question  2:  Are  there  differences  in  reaction  time  attributable  to  runway  separation,  specifically, 
3,000  ft  vs  3,400  ft? 

Findings:  Whether  the  runway  separation  was  3,000  ft  or  3,400  ft,  mean  ART  did  not  differ 

significantly.  However,  an  area  of  difference  and  concern  is  the  effect  of  total 
navigational  system  error  (TNSE)  in  the  3,000-ft  runway  separation  condition.  (For 
details,  see  Section  5.2.) 

Question  3:  When  the  runway  separation  is  3,000  ft,  are  there  differences  in  reaction  time 
attributable  to: 

a.  the  angle  of  the  deviation  of  the  approach  blunder, 

b.  the  range  (nmi  from  the  runway  threshold)  of  aircraft  at  time  of  blunder,  and 

c.  the  type  of  blunder,  specifically,  “Single  Type”  vs  “Fast/Slow  Type?” 

Findings:  a.  No,  no  significant  difference  was  found  to  be  attributable  to  deviation  angle.  (For 

details,  see  Section  5.2.) 

b.  Yes,  when  comparing  approach  blunders  occurring  at  far  range  vs  near  range, 
15-deg  deviation  angle.  Whether  an  approach  blunder  was  presented  at  1.0-s  or 
2.4-s  sensor  update  interval,  the  mean  ART  was  significantly  slower  for  blunders 
at  far  range  than  at  near  range.  Mean  ART  for  far  range  blunders  was  1.7  s 
slower  in  the  1.0-s  sensor  update  interval  condition  and  2.4  s  slower  in  the  2.4-s 
sensor  update  interval.  (For  details,  see  Section  5.3.) 

c.  No,  no  significant  difference  was  found  to  be  attributable  to  blunder  type  in  the 
_ case  of  “Single  Type”  vs  “Fast/Slow  Type.”  (For  details,  see  Section  5.4.) 

Question  4:  Will  controllers  accept  the  PRM  system  as  a  means  of  safely  conducting  independent 
parallel  approaches  during  IFR  conditions  to  parallel  runways  spaced  3,000  ft  apart  ? 

Findings:  Half  of  the  controllers  agreed  that  independent  IFR  approaches  to  runways  separated 

by  3,000  ft  can  be  safely  conducted  using  the  PRM.  However,  half  of  the  controllers 
were  undecided.  In  general,  controllers  voiced  concerns  regarding  the  increased 
workload  and  possible  frequency  congestions  that  may  be  caused  by  the  effect  of 
_ TNSE  experienced  at  this  runway  separation.  (For  details  see  Section  5.5.) _ 


5.1  EFFECT  OF  RUNWAY  SEPARATIONS  AT  BOTH  SENSOR  UPDATE  INTERVALS 


In  order  to  assess  the  effect  of  runway  separation,  the  ART  results  from  Study  I,  “Single 
Type”  approach  blunders,  3,400-ft  runway  separation,  were  compared  to  the  ART  results  from  Study 
II,  “Single  Type"  approach  blunders,  3,000-ft  runway  separation.  The  results  from  both  sensor 
update  intervals  are  presented  in  Table  5-1. 

The  reader  is  cautioned  that  there  are  limits  to  the  comparisons  that  can  be  made  between 
these  two  sets  of  data.  The  two  studies  were  not  the  same  in  all  aspects  expect  for  one  variable,  i.e., 
runway  separation.  Differences  between  the  arrival  push  simulations  used  in  each  of  the  studies  were 
discussed  in  Section  2.2.  One  of  the  main  differences  was  the  increase  in  TNSE  depicted  in  the 
simulations  used  in  Study  n.  In  addition,  Study  I  results  are  based  on  the  responses  of  50  controllers, 
while  Study  II  was  a  small  sample  study  involving  the  responses  of  10  controllers.  Since  Study  II 
results  are  based  on  a  small  sample  size,  findings  are  suggestive  rather  than  definitive. 

The  intention  of  Study  II  was  to  serve  as  a  starting  point  in  studying  the  problem  of  reduced 
separation  in  the  case  of  runway  separation  of  3,000  ft.  It  was  hoped  that  this  small-sample  study 
would  determine  the  feasibility  of  this  type  of  operation  and  might  identify  issues  and  procedures 
that  should  be  resolved  prior  to  implementation.  With  the  limitations  and  intention  of  Study  II  in 
mind,  we  proceed  to  the  findings. 
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Table  5-1 


A  Comparison  of  ART  for  “Single  Type"  Approach  Blunders  at 
Two  Sensor  Update  Intervals  and  Two  Runway  Separations 


Runway  Separation 

3,400  ft 

Update  Interval 

1.0  S 

2.4  s 

1.0  S 

2.4  S 

#  of  Responses 

392 

392 

60 

48 

Maximum  ART 

16.3 

11.5 

6.9 

7.5  i 

Minimum  ART 

-2.8 

-2.7 

0.7 

-0.7 

sd 

1.9 

1.9 

1.2 

1.6 

Mean  ART 

2.7 

2.8 

2.5 

2.6 

The  above  mean  ARTs  indicate  that  whether  the  runway  separation  was  3,400  ft  or  3,000  ft 
and  whether  the  sensor  update  interval  was  1.0-s  or  2.4-s,  controller  reaction  time  remained  basically 
the  same.  Therefore,  when  we  investigate  differences  between  results  obtained  in  response  to  3,000  ft 
vs  3,400  ft,  ART  does  not  appear  to  be  an  area  of  significant  difference.  The  area  of  greatest 
difference  appears  to  be  the  effect  of  TNSE  in  the  3,000-ft  runway  separation  condition.  Due  to  the 
decreased  size  of  the  NOZ,  TNSE  caused  a  number  of  aircraft  to  near  or  enter  the  NTZ.  As  a  result, 
two  types  of  nuisance  alerts  occurred:  1)  Caution  Alerts  on  aircraft  that  then  return  to  course  and  do 
not  penetrate  the  NTZ,  and  2)  Warning  Alerts  on  aircraft  that  skirt  the  NTZ,  penetrate  it  briefly  thus 
setting  off  an  alert,  but  then  return  to  course.  Figure  5-1  depicts  the  two  types  of  nuisance  alerts. 


NON-BLUNDER 
WARNING  ALERT: 
Aircraft  symbol  changes 
to  Red 

CAUTION  ALERT: 
Aircraft  symbol  changes 
to  Yellow 


Figure  5-1.  PRM  Nuisance  Alerts  when  the  runway  separation  is  3,000ft. 


These  deviations  and  the  resulting  nuisance  alerts,  can  result  in  an  increase  in  controller 
workload.  They  may  cause  the  controller  to  issue  numerous  instructions  to  the  pilot  to  return  to 
course,  and  they  may  cause  the  controller  to  break  out  what  appears  to  be  (but  is  not)  an  endangered 
aircraft.  If  controller  workload  is  increased,  one  would  expect  that  the  length  of  time  the  controller 
should  serve  on  the  Monitor  Controller  position  should  be  decreased. 

In  addition,  pilot  workload  becomes  a  concern.  If  a  pilot  is  given  a  go-around  due  to  a  non¬ 
blunder  Warning  Alert,  this  increases  the  workload  of  the  aircrew.  There  is  also  a  monetary  cost  for 
the  additional  time  and  fuel  used.  Nuisance  alerts  also  have  an  effect  on  communication  load.  With 
an  increase  in  the  number  of  communication  transmissions,  frequency  congestion  may  result. 

Table  5-2  lists  the  number  of  non-blunder  Warning  Alerts  and  the  breakouts  which  followed 
the  occurrence  of  these  alerts. 
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Table  5-2 


Non-Blunder  Warning  Alerts  and  Subsequent  Breakouts 
(3,000-ft  Runway  Separation) 


#  of  Pairs  of  Controllers 

5 

Total  #  of  Aircraft 

3160 

Total  #  of  Non-Blunder  Wamina  Alerts 

69 

#  of  Non-Blunder  Warning  Alerts  which 
resulted  in  breakouts 

21 

In  summary,  one  of  the  main  results  of  Study  II  is  the  realization  that  options  for  decreasing 
the  effect  of  TNSE  needs  to  be  explored.  By  decreasing  the  number  of  deviations  described,  there 
should  be  a  subsequent  decrease  in  both  workload  of  controllers  and  aircrews  and  also  cost,  time,  and 
overall  safety  benefits  would  result 

5.2  EFFECT  OF  DEVIATION  ANGLE  AT  BOTH  SENSOR  UPDATE  INTERVALS 

In  order  to  assess  the  effect  of  deviation  angle,  the  responses  of  the  controllers  to  the  “Single 
Type”  approach  blunders  were  analyzed.  ART  results  are  presented  in  Table  5-3. 


Table  5-3 


A  Comparison  of  ART  for  “Single  Type”  Approach  Blunders 
for  Both  Sensor  Update  Intervals  and  Both  Deviation  Angles 
(3,000-ft  Runway  Separation) 


Runway  Separation 

3,000  ft 

Update  Interval 

S 

2.4 

S  _ I 

Deviation  Angle  (deg) 

1  5 

30 

1  5 

■-mTwm 

#  of  Responses 

20 

28 

EH 

28 

Maximum  ART 

3.3 

6.9 

■9 

4.8 

Minimum  ART 

0.7 

1.0 

0.7 

sd 

0.7 

1.4 

1.2 

Mean  ART 

2.0 

2.4 

19 

2.2 

As  discussed  in  Section  4.1.3,  results  of  Study  I  indicate  that  at  both  sensor  update  intervals, 
with  runway  separation  of  3,400  ft,  the  mean  ART  for  approach  blunders  with  30-deg  deviation  angle 
was  approximately  1.0  s  faster  than  for  approach  blunders  with  15-deg  deviation  angle 
(approximately  2.2-2.3  s  vs  3. 1-3.3  s).  This  difference  in  mean  ART  was  found  to  be  significant. 
With  a  30-deg  deviation  angle,  penetration  of  the  NTZ  occurred  more  rapidly  than  with  a  15-deg 
deviation  angle.  Therefore,  there  was  less  time  before  NTZ  penetration,  so  the  controller  reacted 
more  rapidly  to  the  30-deg  blunders.  This  result  was  not  found  in  the  case  of  3,000-ft  runway 
separation. 
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A  paired-sample  t-test  was  performed  in  order  to  assess  the  statistical  significance  of  the 
difference  between  the  mean  ARTs  for  approach  blunders  presented  at  1.0-s  sensor  update  interval, 
15-deg  vs  30-deg  deviation  angles,  3,000-ft  runway  separation.  Results  indicate  that  t  =  .22,  P  =  .828 
(df  =  29).  Since  the  probability  is  >.005,  the  null  hypothesis  is  accepted,  i.e.,  there  is  no  significant 
difference  in  mean  ART  due  to  deviation  angle. 

A  paired-sample  t-test  was  performed  in  order  to  assess  the  statistical  significance  of  the 
difference  between  the  mean  ART  for  approach  blunders  presented  at  2.4-s  sensor  update  interval, 
15-deg  vs  30-deg  deviation  angles,  3,000-ft  runway  separation.  Results  indicate  that  t  =  2.18,  P  = 
.044  ( df  =  17).  Since  the  probability  is  >.005,  the  null  hypothesis  is  accepted;  i.e.,  there  is  no 
significant  difference  in  mean  ART  due  to  blunder  angle. 

In  summary,  while  deviation  angle  was  a  key  factor  in  mean  ART  when  runway  separation 
was  3,400  ft,  it  did  not  result  in  significant  differences  in  mean  ART  when  the  runway  separation  was 
3,000  ft.  This  may  be  caused  by  the  lessened  NOZ  in  the  3,000-ft  situation. 

5.3  EFFECT  OF  APPROACH  BLUNDER  RANGE 

The  effect  of  the  range  at  which  the  approach  blunder  occurred  was  studied  within  approach 
blunders  presented  at  the  two  sensor  update  intervals  and  two  deviation  angles.  ART  results  are 
presented  in  Table  5-4  for  approach  blunders  with  a  15-deg  and  a  30-deg  deviation  angle  when  the 
range  of  the  approach  blunder  was  near  vs  far  from  the  runway  threshold.  ART  tor  both  1.0-s  and 
2.4-s  sensor  update  intervals  in  the  “Single  Type"  of  approach  blunder  are  shown. 


Table  5-4 


A  Comparison  of  ART  for  “Single  Type”  Approach  Blunders, 
15-deg  vs  30-deg  Deviation  Angle,  and  Near  vs  Far  Ranges 
(3,000-ft  Runway  Separation) 


Update  Interval 

1.0 

S 

2.4 

s 

Deviation  Angle  (deg) 

1  5 

30 

1  5 

30 

Blunder  Range 

near 

far 

near 

far 

near 

far 

near 

far 

#  of  Responses 

10 

10 

10 

10 

10 

o 

8 

Maximum  ART 

3.3 

5.8 

6.8 

2.6 

4.7 

3.0 

Minimum  ART 

0.8 

2.2 

1.0 

1.3 

0.2 

■if 

0.7 

sd 

0.7 

1.2 

2.0 

0.4 

1.3 

1.4 

1.2 

Mean  ART 

1.9 

3.6 

3.2 

1.7 

2.0 

■H 

2.1 

2.0 

Four  paired-sample  t-tests  were  performed  to  assess  the  effect  of  range  on  the  ART  to 
approach  blunders  within  a  15-deg  and  a  30-deg  deviation  angle. 
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(1)  1.0-s  sensor  update  interval,  15-deg  deviation  angle,  near  vs  far  range: 

Results  indicate  that  t  =  -4.27,  P  =  .002  {df  =  9).  Since  the  probability  is  <.005,  the  null 
hypothesis  is  rejected;  i.e.,  there  is  a  significant  difference  in  mean  ART  due  to  the 
range  at  which  the  approach  blunder  with  a  15-deg  deviation  angle  occurred  during  the 
1.0-s  sensor  update  interval.  This  is  a  very  strong  effect.  The  probability  of  this  effect 
occurring  by  chance  due  to  sampling  error  is  approximately  2  in  1,000.  In  the  far 
range  approach  blunders,  the  mean  ART  was  1.7  s  longer  than  during  the  near  range 
approach  blunders. 

(2)  1.0-s  sensor  update  interval,  30-deg  deviation  angle,  near  vs  far  range: 

Results  indicate  that  t  =  2.19,  P  =  .056  {df  =  9).  Since  the  probability  is  >.005,  the  null 
hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to  the 
range  at  which  the  approach  blunder  with  a  30-deg  deviation  angle  occurred  during  the 
1.0-s  sensor  update  interval. 

(3)  2.4-s  sensor  update  interval,  15-deg  deviation  angle,  near  vs  far  range: 

Results  indicate  that  t  =  -4.16,  P  =  .002  {df  =  9).  Since  the  probability  is  <.005  the 
null  hypothesis  is  rejected;  i.e.,  there  is  a  significant  difference  in  mean  ART  due  to  the 
ranee  at  which  the  approach  blunder  with  15-deg  deviation  angle  occurred  during  the 
2.4-s  sensor  update  interval.  This  is  a  very  strong  effect.  The  probability  of  this  effect 
occurring  by  chance  due  to  sampling  error  is  approximately  2  in  1,000.  This  result  is 
similar  to  the  results  found  for  the  same  deviation  angle  and  range  in  the  1.0-s  sensor 
update  interval  condition. 

(4)  2.4-s  sensor  update  interval,  30  deg  deviation  angle,  near  vs  far  range: 

Results  indicate  that  t  -  0.12,  P  =  .905  {df  =  7).  Since  the  probability  is  >.005,  the  null 
hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to  the 
range  at  which  the  approach  blunders  with  a  30-deg  deviation  angle  occurred  during 
the  2.4-s  sensor  update  interval. 

In  summary,  whether  an  approach  blunder  was  presented  at  1.0-s  or  2.4-s  sensor  update 
interval,  when  the  deviation  angle  was  15  deg  the  ART  was  significantly  slower  for  blunders  at  far 
ranges  than  the  ART  was  for  approach  blunders  at  near  ranges.  Mean  ART  was  approximately  1.7  s 
slower  in  the  1.0-s  sensor  update  interval  condition  and  approximately  2.4  s  slower  in  the  2.4-s 
sensor  update  interval  condition. 

The  reader  is  cautioned  that  in  the  above  four  paired-sample  t-tests,  the  number  of  responses 
was  quite  small.  Data  were  derived  from  a  small  sample  of  controllers  (8  to  10  controllers)  each 
responding  to  one  approach  blunder  of  the  specific  type  analyzed.  However,  it  is  interesting  to  find 
the  similarity  in  findings  from  Study  I  and  Study  n.  Controllers  reacted  more  slowly  to  the  15-deg 
deviation  angle  approach  blunders  at  far  range  than  to  those  at  near  range.  This  effect  was  the  same 
regardless  of  whether  the  approach  blunder  was  presented  at  1.0-s  or  2.4-s  sensor  update  interval. 

The  rationale  for  this  result  (stated  in  Section  4.1.4)  that  was  given  in  the  case  of  Study  I 
(3,400-ft  runway  separation)  holds  true  in  the  case  of  Study  II  (3,000-ft  runway  separation).  It  may 
be  that  this  effect  occurred  in  the  approach  blunders  with  15-deg  deviation  angle  and  not  in  the 
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30-deg  deviation  angle,  because  with  the  more  severe  deviation  angle  of  30  deg,  the  controllers  know 
that  they  must  act  as  quickly  as  possible  regardless  of  the  range  of  the  blunder.  A  question  is:  Why 
on  the  15 -deg  approach  blunders  would  ART  be  slower  when  the  approach  blunder  occurs  further 
from  the  runway  threshold?  It  may  be  because  at  the  far  range  the  aircraft  are  not  stable  on  the 
localizer,  TNSE  is  greater,  and  more  Caution  Alerts  are  occurring.  The  controllers  may  have  been 
waiting  for  greater  flight  path  stabilization  and,  therefore,  waited  longer  after  the  Caution  Alert  was 
received  at  the  far  range  vs  near  range. 

5.4  EFFECT  OF  APPROACH  BLUNDER  TYPE 

In  order  to  assess  the  difference  in  ART  which  may  have  been  attributed  to  the  approach 
blunder  type,  two  approach  blunder  types  were  studied:  “Single  Type”  and  “Fast/Slow  Type” 
approach  blunder  (see  Section  2.1.1  for  approach  blunder  definitions).  Tables  5-5  through  5-8 
provide  summaries  of  ART  results  on  these  two  types  of  approach  blunders.  Four  paired-sample 
t-tests  were  performed  in  order  to  compare  these  two  approach  blunder  types  and  results  are  provided 
after  each  table. 

(1)  1.0-s  sensor  update  interval,  15-deg  deviation  angle,  near  range: 


Table  5-5 


A  Comparison  of  ART  for  "Single  Type"  vs  "Fast/Slow  Type”  Approach  Blunders, 
1.0-s  Sensor  Update  interval,  15-deg  Deviation  Angle,  Near  Range 
(3,000-ft  Runway  Separation) 


Blunder  Type 

“Slnale” 

“Fast/Slow” 

#  of  Responses 

10 

10 

Maximum  ART 

3.3 

7.1 

Minimum  ART 

0.8 

0.8 

sd 

0.7 

1.8 

Mean  ART 

1.9 

2.9 

Results  indicate  that  t  =  -2.59,  P  -  .029  (df  -  9).  Since  the  probability  is  >.005,  the 
null  hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
blunder  type.  (See  Section  4.1.5). 
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(2)  2.4-s  sensor  update  interval,  15-deg  deviation  angle,  near  range: 


Table  5-6 


A  Comparison  of  ART  for  “Single  Type”  vs  “Fast/Slow  Type”  Approach  Blunders, 
2.4-s  Sensor  Update  Interval,  15-deg  Deviation  Angie,  Near  Range 
(3,000-ft  Runway  Separation) 


Blunder  Type 

“Sinole” 

“Fast/Slow” 

#  of  Responses 

10 

10 

Maximum  ART 

5.8 

7.9 

Minimum  ART 

2.2 

3.0 

sd 

1.2 

1.4 

Mean  ART 

3.6 

4.2 

Results  indicate  that  t  ~  -0.98,  P  =  .355  (df  =  9).  Since  the  probability  is  >.005,  the  null 
hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
approach  blunder  type. 

(3)  1.0-s  sensor  update  interval,  15-deg  deviation  angle,  far  range: 

Table  5-7 


A  Comparison  of  ART  for  “Single  Type”  vs  “Fast/Slow  Type”  Approach  Blunders, 
1.0-s  Sensor  Update  Interval,  15-deg  Deviation  Angle,  Far  Range 
(3,000-ft  Runway  Separation) 


Blunder  Type 

“Sinale” 

“Fast/Slow” 

#  of  Responses 

9 

9 

Maximum  ART 

4.7 

3.5 

Minimum  ART 

0.2 

0.7 

sd 

1.3 

1.1 

Mean  ART 

2.2 

2.1 

Results  indicate  that  t  =  -.09,  P  =  .927  (df  =  8).  Since  the  probability  is  >.005,  the  null 
hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
approach  blunder  type. 
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(4)  2.4-s  sensor  update  interval,  15-deg  deviation  angle,  far  range: 


Table  5-8 


A  Comparison  of  ART  for  “Single  Type"  vs  “Fast/Slow  Type”  Approach  Blunders, 
2.4-s  Sensor  Update  Interval,  15-deg  Deviation  Angle,  Far  Range 
(3,000-ft  Runway  Separation) 


Blunder  Type 

“Slnale" 

“Fast/Slow” 

#  of  Responses 

9 

9 

Maximum  ART 

7.6 

6.3 

Minimum  ART 

2.4 

1.6 

sd 

1.9 

1.3 

Mean  ART 

4.4 

3.0 

Results  indicate  that  t  =  1.73,  P  =  .118  (df  =  9).  Since  the  probability  is  >.005,  the  null 
hypothesis  is  accepted;  i.e.,  there  is  no  significant  difference  in  mean  ART  due  to 
approach  blunder  type. 

Whether  the  approach  blunder  type  was  “Single  Type”  or  “Fast/Slow  Type,”  when 
comparing  ART  from  blunders  presented  at  the  same  sensor  update  interval,  deviation  angle,  and 
range,  there  is  no  significant  difference  in  the  ART  due  to  approach  blunder  type. 

5.5  CONTROLLER  OPINION  ON  THE  USE  OF  THE  PRM  WHEN  RUNWAY 

SEPARATION  IS  3,000  FT 

Controller  opinion  survey  forms  were  used  to  obtain  the  opinions  of  the  controllers  on  the 
effectiveness  of  the  PRM  system  and  its  overall  acceptability  for  use.  Table  5-9  lists  the  percentage  of 
controllers  who  agreed,  disagreed,  or  were  undecided  regarding  each  survey  statement  The  complete 
text  of  each  survey  statement  is  also  presented,  accompanied  by  a  summary  of  results  and  any 
narrative  comments  made  by  the  controllers. 
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Table  5-9 


Study  II 

Summary  of  Controller  Opinion  Survey  Results 


Survey  Item 

■n 

2.0 

GENERAL  ACCEPTANCE 

2.1 

Approaches  with  runways  separated  by  3,000  ft 

50 

0 

50 

can  be  safely  conducted 

3.0 

TRAINING 

3.1 

Training  time  was  adequate 

100 

0 

0 

3.2 

All  information  was  provided 

100 

0 

0 

4.0 

SIMULATION 

4.1 

Simulated  traffic  density  was  realistic 

90 

10 

0 

4.2 

Simulated  blunder  trajectories  were  realistic 

60 

20 

20 

4.3 

Audio  portion  of  simulation  was  realistic 

70 

20 

10 

Survey  Section  2:  General  Acceptance 

Statement  2.1  Independent  IFR  approaches  to  runways  separated  by  3,000  ft  can  be  safely 
conducted  using  the  PRM. 

Half  of  the  controllers  agreed  with  this  statement  and  half  were  undecided.  Many  of  the 
controllers  stated  that  operations  can  or  might  be  safely  conducted  with  1.0-s  sensor  update  interval. 
Their  opinion  was  divided  on  the  use  of  2.4-s  sensor  update  interval.  They  were  concerns  that  with 
the  decreased  runway  separation  (decreased  from  3,400  to  3,000  ft),  the  faster  update  interval  might 
be  necessary. 

There  were  also  concerns  that  with  the  decreased  runway  separation,  i.e.,  decreased  NOZ,  the 
effect  of  TNSE  is  exacerbated  and  might  result  in  more  unnecessary  breakouts.  Some  controllers 
also  voiced  concern  regarding  the  amount  of  time  on  the  position.  This  is  discussed  further  under 
“Additional  Comments”  made  by  the  controllers. 

Survey  Section  3:  Training 

Statement  3.1  Adequate  training  time  was  provided  to  become  familiar  with  the  display 
before  beginning  the  testing. 

All  controllers  agreed  with  this  statement.  As  the  reader  will  recall,  these  subjects  also 
participated  in  Study  I  and  were,  therefore,  familiar  with  the  PRM  system. 

Statement  3.2  All  information  needed  to  aid  me  in  performing  the  monitoring  task  was 
provided. 

All  controllers  agreed  with  this  statement. 

Survey  Section  4:  The  Simulation 

Statement  4.1  The  simulated  traffic  density  was  realistic. 
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Nine  out  of  the  ten  controllers  agreed  with  this  statement.  The  controller  who  disagreed 
commented  that  the  density  was  sufficient  as  far  as  the  number  of  aircraft  involved  on  the  final 
approach,  but  that  more  transmissions  were  needed  to  be  mixed  into  the  audio. 

Statement  4,2  The  simulated  aircraft  “blunder”  trajectories  were  realistic. 

Six  controllers  agreed,  two  disagree,  and  two  were  undecided.  Similar  to  the  responses  given 
in  Study  I  to  this  question,  most  controllers  who  disagreed  or  were  undecided  gave  one  of  the 
following  reasons:  1)  Some  controllers  said  that  it  was  difficult  to  adjust  to  the  magnification  of  the  x 
and  y  axis.  The  magnification  makes  the  angle  of  the  deviation  appear  more  severe  than  it  actually  is. 
2)  Some  controllers  said  that  there  were  too  many  emergencies  and  that  this  was  unrealistic.  The 
simulation  did  intentionally  show  many  more  blunders  than  one  would  experience  in  actual 
operations.  Actual  blunders  are  infrequent  and,  therefore,  difficult  to  study.  Through  simulation,  a 
number  and  variety  of  blunders  were  presented  in  order  to  obtain  valuable  data  on  controller 
responses. 

Statement  4.3  The  audio  portion  of  the  simulation  was  realistic. 

Seven  controllers  agreed,  two  disagreed,  and  one  was  undecided.  Two  controllers  said  that 
there  needed  to  be  added  communications.  One  controller  mentioned  that  it  was  monotonous 
hearing  the  same  two  pre-recorded  voices  throughout  the  study. 

Survey  Section  5:  Additional  Comments 

Controllers  were  asked  to  comment  on  any  changes  to  the  simulation  which  might  enhance  its 
realism.  Controllers  suggested  that  more  speed  changes  and  additional  traffic  might  be  included  in 
the  simulation.  One  controller  mentioned  that  the  simulations  used  in  this  study  did  not  include 
turbulent  flight  path  conditions,  which  had  been  included  in  Study  I.  Two  controllers  recommended 
further  testing  with  wind  to  consider  its  effect  with  the  reduced  runway  separation. 

Controllers  were  asked  to  identify  any  factors  in  the  simulation  which  might  have  affected  the 
quality  of  the  reaction  time  measurement.  One  controller  recommended  that  the  scope  be  tilted  for 
easier  scanning.  Several  controllers  mentioned  that  the  length  of  time  on  position  may  have  affected 
their  response  time.  Each  simulated  monitoring  session  was  about  45  to  75  minutes  in  length.  Some 
controllers  recommend  the  time  on  position  should  be  no  more  than  60  minutes,  and  perhaps  less. 
One  controller  mentioned  that  in  Atlanta  they  are  required  to  write  down  all  aircraft  IDs  in  case  of  an 
equipment  failure  and,  so  they  would  have  that  information.  This  requires  attention  to  be  diverted 
away  from  the  radar  screen.  This  was  not  required  during  the  study.  If  this  requirement  is  present 
during  implementation  of  this  new  equipment  in  the  field,  this  added  task  should  be  considered  in 
assessing  controller  response  time. 

Controllers  were  asked  to  make  any  additional  comments  regarding  the  simulation,  the 
display,  or  the  study  procedures.  Some  concerns  were  voiced  by  controllers.  The  greatest  concern 
seemed  to  be  workload  and  time  on  position.  With  the  decreased  NOZ  controllers  found  that  they 
had  to  increase  their  attention  to  every  possible  deviation.  There  was  also  concern  that  the  faster 
update  interval  of  1.0  s  would  be  necessary  vs  the  2.4-s  sensor  update  interval.  Controllers  feared 
making  too  many  unnecessary  break-outs  with  the  slower  update  interval  in  use.  The  need  for 
training  and  new  procedures  was  mentioned  as  being  necessary,  if  decreased  separation  is  to  be 
accepted. 
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6.  CONCLUSIONS  AND  RECOMMENDATIONS 

Conclusions  on  Alert  Response  Time  (ART)  derived  from  Study  I  are  discussed  in 
Section  6.1.  Conclusions  on  ART  derived  from  Study  II  are  discussed  in  Section  6.2.  Controller 
opinion  on  use  of  the  PRM  is  discussed  in  Section  6.3.  Recommendations  for  future  research  are 
included  in  Section  6.4. 

6.1  STUDY  I 


6.1.1  Conclusions  on  1.0-s  and  2.4-s  Sensor  Update  Interval,  3,400-ft  Runway 
Separation 

Table  6-1  provides  a  quick  reference  of  the  results  of  the  comparisons  performed.  An 
asterisk  indicates  probability  values  which  showed  a  significant  difference  in  mean  ART.  Reference  is 
made  to  the  section  in  which  the  analysis  is  discussed  in  detail. 
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Table  6-1 


Comparisons  Performed  In  Study  I 
1.0-s  and  2.4-s  Sensor  Update  Intervals,  3,400-ft  Runway  Separation 


Section 

1  Within-Subject  Comparisons  i 

Probability  Values 

4.1.1 

EM  ,1-H 

1 1.0  s  vs  2.4  s  ! 

.309  | 

4.1.3 

r  i'ii  n1  i,i  Ta  ii  n  v\  i  ''1 1 1  i— ii 

1.0-s  sensor  update  interval 

.000*  | 

2.4-s  sensor  update  interval 

.000* 

4.1.4 

1.0-s  sensor  update  interval,  15-deg  deviation  angle 

.001* 

1 ,0-s  sensor  update  interval,  30-deg  deviation  angle 

.143 

2.4-s  sensor  update  interval,  15-deg  deviation  angle 

*  I  *  I  * 

2.4-s  sensor  update  interval,  30-dea  deviation  angle 

.579 

4.1.5 

1  a  w  r  i  ;A'  1  iT »f»i  <  >  1 1  <  w  i  1 1 1  ■  iv--!  i » 1 1  •; 

(30-deg  deviation  angle,  near  and  far  range  pooled) 

1.0-s  sensor  update  interval 

.513 

2.4-s  sensor  update  interval 

.715 

(15-deg  deviation  angle) 

1 .0-s  sensor  update  interval,  near  range 

.670 

1 .0-s  sensor  update  interval,  far  range 

.005 

2.4-s  sensor  update  interval,  near  range 

.639 

2.4-s  sensor  update  interval,  far  range 

.242 

4.1.6 

rm  mi  imi  ini'  iii  ii  ini'  ii  mil  1 1  — 

(15-deg  deviation  angle,  calm  flight  path) 

1 .0-s  sensor  update  interval,  near  range 

.605 

2.4-s  sensor  update  interval,  near  range 

.201 

1.0-s  sensor  update  interval,  far  range 

.079 

2.4-s  sensor  update  interval,  far  range 

.738 

(Far  range,  calm  flight  path) 

1.0-s  sensor  update  interval,  15-deg  deviation  angle 

.070 

2.4-s  sensor  update  interval,  15-deg  deviation  angle 

.003 

1.0-s  sensor  update  interval,  30-deg  deviation  angle 

.077 

2.4-s  sensor  update  interval,  30-deg  deviation  angle 

.895 

4. 1.6.1 

U  [*  V  if  1 i  U.JJNI  •  J*  b  1  l-l 

interval 

(15-deg  deviation,  calm  flight  path) 

near  range 

.026 

farranae 

1  .058 

Between-Sublect  Comparisons 

4.1.7 

EMM  Jloiw  •  mm 

1  novice  vs  experienced  monitor  controller 

.000** 

*  If  the  probability  value  is  .001  or  less,  the  difference  between  the  means  is  considered  significant. 
*#  If  the  probability  value  is  .05  or  less,  the  difference  between  the  means  is  considered  significant. 
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The  following  conclusions  are  made  based  on  the  analyses  summarized  in  the  above  table: 

(1)  Sensor  Update  Interval  -  Regardless  of  sensor  update  interval,  ART  remained  basically 
the  same  (less  than  3.0  s).  Under  both  conditions,  controllers  generally  broke  out  the 
endangered  aircraft  before  the  blundering  aircraft  penetrated  the  NTZ.  Based  on 
application  of  the  risk  assessment  model  [3],  the  conclusion  can  be  made  that  either  of 
the  sensor  update  intervals  provides  acceptable  levels  of  safety.  However,  as  discussed 
in  Section  4.1.1,  an  advantage  of  the  faster  update  interval  is  that  it  provides  increased 
advance  warning  time  which  translates  into  a  greater  miss  distance  between  aircraft.  A 
second  advantage,  discussed  in  Section  4.1.8,  is  a  lower  rate  of  false  breakouts  that 
translates  into  less  arrival  delays. 

(2)  Deviation  Angle  —  One  of  the  most  significant  findings  was  in  the  area  of  deviation 
angle.  In  response  to  both  sensor  update  intervals,  controllers  generally  responded 
more  quickly  (approximately  1.0  s  quicker)  to  deviations  of  greater  angle.  As 
discussed  in  Section  4.1.3,  with  a  30-deg  blunder,  penetration  of  the  NTZ  occurred 
more  rapidly  than  with  a  15-deg  blunder.  There  was  less  time  before  NTZ  penetration 
and  so  the  controller  reacted  more  rapidly  in  response  to  the  severity  of  the  situation. 

(3)  Range  -  Another  very  significant  finding  was  in  the  area  of  range.  During  simulated 
approach  blunders  with  15-deg  deviation  angle,  presented  at  either  sensor  update 
interval,  controllers  generally  reacted  more  slowly  (approximately  1.0  s  slower)  to 
approach  blunders  which  occurred  at  the  far  vs  near  range.  As  discussed  in  Section 
4.1.4,  it  appears  that  controllers  were  more  tolerant  of  deviations  at  the  far  range  since 
they  expected  deviations  (and  the  resulting  Caution  Alerts)  to  occur  due  to  the  aircraft 
instability  on  the  localizer.  In  response  to  30-deg  approach  blunders,  controllers 
reacted  as  quickly  as  possible  and  did  not  delay  a  decision,  regardless  of  the  range  of 
the  deviating  aircraft.  Again,  it  appears  that  the  severity  of  the  deviation  is  the  driving 
force. 

(4)  Flight  Path  Condition  -  Mean  ART  did  not  differ  significantly  as  a  result  of  flight  path 
condition.  Based  on  this  finding,  one  may  conclude  that  when  the  runway  separation 
was  3,400  ft  and  the  PRM  was  used,  response  time  remained  acceptable  whether  calm  or 
turbulent  conditions  were  experienced.  This  conclusion  is,  of  course,  limited  to  the 
particular  parameters  of  “turbulent  condition"  modeled  in  this  simulation. 

(5)  Approach  Blunder  Type  -  It  was  found  that  there  was  no  significant  difference  in  ART 
in  the  "Single  Type”  vs  “Fast/Slow  Type”  approach  blunders.  In  this  case,  it  does  not 
appear  that  speed  of  the  deviating  aircraft  was  the  major  factor  in  decision-making. 
The  major  driving  force  in  accelerating  the  decision  to  breakout  an  aircraft  appears  to 
have  been  the  deviation  angle. 

(6)  Controller  Experience  Level  -  It  was  found  that  experienced  Monitor  Controller 
responded  1.0  s  slower  than  the  controllers  who  did  not  have  experience  in  the  position. 
It  was  originally  thought  that  experienced  Monitor  Controllers  would  be  faster  than  the 
novice  group.  However,  it  appears  that  knowledge  of  the  complexity  of  the  task  and  the 
various  data  to  consider  in  making  a  decision,  caused  the  more  experienced  controller 
to  react  more  slowly  and  with  greater  caution. 
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6.1.2  Conclusions  on  4.8-s  Sensor  Update  Interval,  3,400-ft  Runway  Separation 

Table  6-2  provides  a  quick  reference  of  the  results  of  the  comparisons  performed.  An 
asterisk  indicates  probability  values  which  showed  a  significant  difference  in  mean  ART.  Reference  is 
made  to  the  section  in  which  the  analysis  is  discussed  in  detail. 

Table  6-2 


Comparisons  Performed  In  Study  I 
4.8-s  Sensor  Update  Interval,  3,400-ft  Runway  Separation 


Section 

Withln-Subject  Comparisons 

Probability  Values 

4.2.1 

4.8-s  vs  1 .0-s  sensor  update  interval 

.001* 

4.8-s  vs  2.4-s  sensor  update  interval 

.005 

Ena 

1  ll'llil'lfllllMHH— — i 

Hi 

15  deg  vs  30  deg 

.508 

*  If  the  probability  value  is  .001  or  less,  the  difference  between  the  means  is  considered  significant. 


As  indicated  in  Table  6-2  the  variable  in  which  mean  ART  was  found  to  differ  significantly 
was  sensor  update  interval.  The  stringent  criteria  set  through  application  of  the  Bonfenroni 
Adjustment  was  met  in  the  case  of  4.8-s  vs  1.0-s  sensor  update  interval.  In  the  case  of  4.8-s  vs  2.4-s 
sensor  update  interval,  a  strong  trend  towards  significance  was  demonstrated.  Controller  ART  was 
significantly  slower  (by  approximately  1.0  s)  when  the  4.8-s  sensor  update  interval  was  used  vs  the 
other  two  faster  update  intervals.  Whether  the  deviation  angle  was  15  deg  or  30  deg  the  ART 
remained  at  approximately  4.0  s.  As  discussed  in  Section  4.2.1.  based  on  preliminary  findings 
presented  to  the  Working  Group,  the  group  considered  this  sensor  update  interval  to  be  too  slow  for 
the  operation  to  be  safely  conducted  with  3,400-ft  runway  separation.  Subsequent  testing  was 
therefore  conducted  to  explore  the  case  of  4.8-s  sensor  update  interval  and  4,300-ft  runway 
separation. 

6.1.3  Conclusions  on  4.8-s  Sensor  Update  Interval,  '3,400-ft  vs  4,300-ft  Runway 
Separation 

This  section  includes  the  conclusions  reached  from  the  results  obtained  from  the  simulations 
of  approach  blunders  presented  at  4.8-s  sensor  update  interval,  3,400-ft  vs  4,300-ft  runway 
separation.  The  4,300-ft  separation  case  was  tested  to  gain  some  understanding  of  conducting  the 
simultaneous  independent  approaches  at  today's  standard  runway  separation  of  a  minimum  of 
4,300  ft.  In  this  testing  configuration  the  controllers  had  the  advantage  of  the  PRM  high-resolution, 
color  display  with  automatic  alerting  and  the  advantage  of  greater  radar  accuracy  as  compared  to  the 
current  system  in  use. 

Controller  ART  was  significantly  faster  when  the  runway  separation  was  increased  from 
3,400  ft  to  4,300  ft  (4.2  s  at  3,400-ft  separation  vs  1.6  s  at  4,300-ft  separation).  The  results  were 
highly  significant.  As  discussed  in  Section  4.3,  one  possible  explanation  for  the  extreme  difference 
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in  mean  ART  concerns  the  presence  of  a  larger  normal  operating  zone  (NOZ)  in  the  4,300-ft  runway 
separation  condition.  The  controllers  had  the  opportunity  to  observe  the  blundering  aircraft  for  one 
or  two  more  radar  intervals  prior  to  NTZ  penetration.  When  the  Caution  Alert  sounded,  the  aircraft 
was  far  from  the  centerline.  There  was  little  likelihood  that  this  was  a  deviation  caused  by  TNSE  and 
therefore  the  controller  was  confident  in  making  the  breakout-decision  and  responded  quickly. 

The  occurrence  of  these  rapid  response  times  appears  to  indicate  that  the  operation  could  be 
successfully  performed  at  this  separation  when  the  PRM  is  used.  This  conclusion  was  confirmed  by 
the  application  of  the  risk  assessment  model  [5].  The  PRM  blunder  resolution  performance  was 
tested  at  a  4,300-ft  runway  separation  for  the  “least  favorable”  condition:  a  4.8-s  sensor  update 
interval,  30-deg  blunders,  far  range.  The  percent  of  trials  with  miss  distances  under  500  ft  was  0.32% 
at  1-milliradian  accuracy  and  0.310%  at  2-milliradian  accuracy.  The  4,300-ft/4.8-s  probability  of 
0.32%  is  very  similar  to  the  results  on  3,400-ft/l.O-s  sensor  update  interval  and  3,400-ft/2.4-s  sensor 
update  interval  [5]. 

6.1.4  Additional  Conclusions  on  4,300-ft  Runway  Separation 

Regarding  the  simulations  showing  4,300-ft  separation,  Table  6-3  provides  a  quick  reference 
of  the  results  of  the  comparisons  perfoimed  in  Section  4.3.  An  asterisk  indicates  probability  values 
which  showed  a  significant  difference  in  mean  ART.  Reference  is  made  to  the  section  in  which  the 
analysis  is  discussed  in  detail. 


Table  6-3 


Comparisons  Performed  in  Study  I 
4.8-s  Sensor  Update  Interval,  4,300-ft  Runway  Separation 


Section 

Wlthin-Subject  Comparisons 

Probability  Values 

4.4.1 

.121 

15  deg  vs  30  deg 

mm 

.000* 

30-deg  deviation  angle,  calm  flight  path 

mm 

.895 

15-deg  deviation  angle,  near  range,  calm  flight  path 

4.4.4 

.875 

15-deg  deviation  angle,  near  range,  calm  flight  path 

*  If  the  probability  value  is  .001  or  less,  the  difference  between  the  means  is  considered  significant. 


As  indicated  in  Table  6-3,  the  variable  in  which  ART  was  found  to  differ  significantly  was 
blunder  range.  As  discussed  in  Section  4.4.2,  ART  in  response  to  approach  blunders  at  far  range 
tends  to  be  longer  since  TNSE  is  greater  at  far  vs  near  range  and  the  controllers  seemed  to  wait  for 
the  aircraft  to  stabilize  on  the  localizer  before  making  a  breakout  decision. 
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6.2  STUDY  H 


Table  6-4  provides  a  quick  reference  of  the  results  of  the  comparisons  performed  in 
Section  5.  An  asterisk  indicates  probability  values  which  showed  a  significant  difference  in  mean 
ART.  Reference  is  made  to  the  section  in  which  the  analysis  is  discussed  in  detail. 

Table  6-4 

Comparisons  Performed  in  Study  II 
1.0-s  and  2.4-s  Sensor  Update  Interval,  3,000-ft  Runway  Separation 


Section 

Wlthln-Subject  Comparisons 

Probability  Values 

5.2 

1.0-s  sensor  update  interval 

.828 

2.4-s  sensor  update  interval 

.044 

5.3 

1.0-s  sensor  update  interval,  15-deg  deviation  angle 

.002* 

1 .0-s  sensor  update  interval,  30-deg  deviation  angle 

.056 

2.4-s  sensor  update  interval,  15-deg  deviation  angle 

.002* 

2.4-s  sensor  update  interval,  30-deg  deviation  angle 

.905 

5.4 

(15-deg  deviation  angle) 

.029 

1.0-s  sensor  update  interval,  near  range 

.355 

2.4-s  sensor  update  interval,  near  range 

.927 

1.0-s  sensor  update  interval,  far  range 

2.4-s  sensor  update  interval,  far  range 

.118 

*  If  the  probability  value  is  .005  or  less,  the  difference  between  the  means  is  considered  significant. 


Findings  regarding  the  effect  of  sensor  update  interval  (1.0  s  vs  2.4  s)  were  similar  to  those  of 
Study  I.  Overall,  ART  did  not  differ  significantly  on  the  basis  of  sensor  update  interval.  Regarding 
blunder  range,  in  response  to  approach  blunders  with  a  15-deg  deviation  angle,  controllers  tended  to 
respond  more  slowly  to  blunders  occurring  at  the  far  range.  As  in  Study  I,  controllers  were  probably 
waiting  for  the  aircraft  at  the  far  range  to  stabilize  on  the  localizer  before  making  a  breakout 
decision. 

A  major  concern  which  surfaced  in  Study  II  was  the  effect  of  TNSE  due  to  the  closeness  of 
the  runways  in  the  3,000-ft  separation  condition.  The  effect  of  TNSE  seems  to  have  caused 
controllers  to  question  the  safety  of  this  operation,  regardless  of  the  speed  of  their  ART.  The  main 
result  of  Study  II  was  the  realization  of  the  need  for  consideration  of  means  to  lessen  the  effect  of 
TNSE,  thus  decreasing  the  frequency  and  magnitude  of  deviations  and  NTZ  penetrations.  These 
means  should  be  applied  through  simulation  study,  before  a  decision  can  be  made  regarding  safety. 
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6.3  CONTROLLER  OPINION  -  STUDY  I  AND  H 

Controllers  were  very  enthusiastic  about  the  PRM.  They  stated  that  through  use  of  this 
equipment,  the  standard  runway  spacing  could  be  reduced  to  the  3,400-ft  spacing  demonstrated. 
Controllers  unanimously  agreed  that  the  PRM  was  superior  to  the  Automated  Radar  Terminal  System 
(ARTS).  They  especially  liked  the  high-resolution  color  display  and  automated  alerts.  All,  but  two 
controllers  agreed  that  approaches  could  be  safely  conducted  at  3,400-ft  spacing,  if  monitored  by  the 
PRM.  The  two  controllers  wished  to  reserve  their  opinion  until  they  had  more  familiarity  with  the 
system. 

Regarding  3,000-ft  spacing,  all  controllers  were  not  of  the  opinion  that  the  operation  could  be 
safely  conducted  with  PRM.  Five  of  the  ten  controllers  thought  that  operations  can  be  safely 
conducted  with  1.0-s  sensor  update  interval.  They  were  not  sure  about  the  safety  of  use  of  2.4-s 
sensor  update  interval.  As  discussed  above,  it  appears  that  controllers  were  hesitant  due  to  the  effect 
of  TNSE  which  occurs  at  the  lesser  runway  spacing  of  3,000  ft  Controllers  recommended  that  means 
be  explored  to  reduce  the  effect  of  TNSE  or  else  it  would  be  necessary  to  limit  the  controller’s  time 
on  position.  It  was  believed  that,  due  to  the  high  number  of  non-blunder  alerts,  controllers  would 
become  fatigued  at  this  position  within  approximately  30  to  60  minutes. 

6.4  RECOMMENDATIONS 

With  the  benefit  of  20/20  hindsight,  there  are  elements  of  the  experiment  that  we  would  do 
differently,  given  the  opportunity  to  conduct  further  experimentation.  In  order  to  enhance  the 
realism  of  the  simulation,  the  noise  level  representative  of  an  actual  TRACON  would  be  included 
during  the  simulations.  In  addition,  changes  would  be  made  in  the  pre-recorded  audio  of  the  dialog 
between  controller  and  pilots.  In  the  pre-recorded  audio  which  accompanied  each  visual  presentation 
of  an  arrival  push,  one  controller  and  one  pilot  were  used.  For  future  studies,  a  voice  synthesizer  is 
recommended  so  that  the  pre-recorded  audio  will  sound  like  various  pilots  and  will  therefore  be  more 
realistic. 

Further  study  needs  to  be  performed  regarding  3,000-ft  spacing.  Ways  to  alleviate  the  effect 
of  TNSE  need  to  be  explored. 

One  such  study  has  recently  been  conducted  at  the  FAA  Technical  Center,  through  the 
collaborative  efforts  of  The  Multiple  Parallel  Technical  Work  Group,  the  FAA  Technical  Center,  and 
CTA  Incorporated.  The  study  examined  dual  parallel  runways  spaced  3,000  ft  apart  with  one  ILS 
approach  localizer  offset  by  1  deg.  Having  one  runway  with  a  1-deg  localizer  offset  increases  the 
NOZ  for  each  runway.  With  the  increased  NOZ,  there  is  a  reduction  in  the  effect  of  TNSE  and, 
therefore,  a  reduction  in  the  number  of  NTZ  entries  and  accompanying  nuisance  alerts.  A  report  of 
findings  is  expected  in  the  near  future. 

In  our  studies  the  look-ahead  time  for  the  Caution  Alert  was  10  s.  The  effect  of  various  look¬ 
ahead  times  should  be  explored  in  scenarios  depicting  runway  separations  of  interest.  We  need  to 
explore  the  trade-off  of  having  more  notice  vs  the  possibility  of  increasing  the  number  of  false  alerts 
and  subsequent  false  breakouts. 

We  should  also  take  another  look  at  the  predictor  line.  Controllers  were  concerned  at  how 
erratic  it  appeared  when  the  sensor  update  interval  was  as  fast  as  1.0  s.  Although  we  would  like  to  see 
a  smoother  movement  for  the  predictor  line,  i.e.,  less  erratic  jumping  and  greater  consistency  of 
position,  we  do  not  want  to  have  substantial  losses  in  the  accuracy  of  prediction. 
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APPENDIX  A 


Arrival  Pushes  Shown  In  Study  I,  Weeks  1  through  12 


Arrival  Push 

Blunder  Number 
and  Type 

Update 

Interval 

(sec) 

Runway 

Separation 

(ft) 

Angle 

(deg) 

Range 

(nml) 

Fit.  Path 
Conditions 

lAand  IB 

11. 

Distraction 

1.0 

3,400 

15 

far 

Calm 

5. 

Single 

1.0 

3,400 

30 

near 

Calm 

13. 

Simu  It/Miss 

1.0 

3,400 

15 

0.5 

Calm 

12. 

Distraction 

1.0 

3,400 

30 

far 

Calm 

7. 

Single 

1.0 

3,400 

30 

far 

Calm 

2A  and  2B 

10. 

Fast/Slow 

1.0 

3,400 

15 

far 

Calm 

3. 

Single 

1.0 

3,400 

15 

far 

Calm 

1. 

Single 

1.0 

3,400 

15 

near 

Calm 

9. 

Fast/Slow 

1.0 

3,400 

15 

near 

Calm 

3A  and  3B 

8. 

Single 

1.0 

3,400 

30 

far 

Turbulent 

4. 

Single 

1.0 

3,400 

15 

far 

Turbulent 

6. 

Single 

1.0 

3,400 

30 

near 

Turbulent 

2. 

Single 

1.0 

3,400 

15 

near 

Turbulent 

4A  and  4B 

3. 

Single 

2.4 

3,400 

15 

far 

Calm 

5. 

Single 

2.4 

30 

near 

Calm 

10. 

Fast/Slow 

2.4 

3,400 

15 

far 

Calm 

12. 

Distraction 

2.4 

3,400 

30 

far 

Calm 

9. 

Fast/Slow 

2.4 

3,400 

15 

near 

Calm 

5A  and  5B 

1. 

Single 

2.4 

3,400 

15 

near 

Calm 

13. 

Simult/Miss 

2.4 

3,400 

15 

0.5 

Calm 

7. 

Single 

2.4 

3,400 

30 

far 

Calm 

11. 

Distraction 

2.4 

3,400 

15 

far 

Calm 

6A  and  6B 

6. 

Single 

2.4 

3,400 

30 

near 

Turbulent 

4. 

Single 

2.4 

3,400 

15 

far 

Turbulent 

8. 

Single 

2.4 

3,400 

30 

far 

Turbulent 

2. 

Single 

2.4 

3,400 

15 

near 

Turbulent 

7A  and  7B 

5. 

Single 

4.8 

3,400 

30 

far 

Calm 

8. 

Fast/Slow 

4.8 

3,400 

15 

far 

Calm 

1. 

Single 

4.8 

3,400 

15 

near 

Calm 

9. 

Distraction 

4.8 

3,400 

15 

far 

Calm 

8A  and  8B 

3. 

Single 

4.8 

3,400 

30 

near 

Calm 

10. 

Distraction 

4.8 

3,400 

30 

far 

Calm 

11. 

Simult/Miss 

4.8 

3,400 

15 

0.5 

Calm 

7. 

Fast/Slow 

4.8 

3,400 

15 

near 

Calm 

9A  and  9B 

4. 

Single 

4.8 

3,400 

30 

near 

Turbulent 

6. 

Single 

4.8 

3,400 

30 

far 

Turbulent 

2. 

Single 

4.8 

3,400 

15 

near 

Turbulent 
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APPENDIX  B 


Arrival  Pushes  Shown  In  Study  I,  Weeks  13  through  25 


Note:  The  only  difference  between  the  arrival  pushes  presented  in  Weeks  1-12  and  Weeks  13-25  is  the 
change  in  runway  separation  in  the  4.8-s  sensor  update  interval  scenarios.  They  were  changed 
from  3,400  ft  in  Weeks  1-12  to  4,300  ft  in  Weeks  13-25. 


Arrival  Push 

Blunder  Number 
and  Type 

Update 

Interval 

(sec) 

Runway 

Separation 

ill) _ 

Angle 

(deg) 

Range 

(nml) 

Fit.  Path 
Conditions 

lAand  IB 

11. 

Distraction 

1.0 

HRH 

15 

far 

Calm 

5. 

Single 

1.0 

Be 

30 

near 

Calm 

13. 

Simu  It/Miss 

1.0 

CV 

15 

0.5 

Calm 

12. 

Distraction 

1.0 

30 

far 

Calm 

7. 

Single 

1.0 

30 

far 

Calm 

2A  and  2B 

10. 

Fast/Slow 

1.0 

15 

far 

Calm 

3. 

Single 

1.0 

Bt  •  life' 

15 

far 

Calm 

1. 

Single 

1.0 

(cjr  •  Myayafl 

15 

near 

Calm 

9. 

Fast/Slow 

1.0 

15 

near 

Calm 

3A  and  3B 

8. 

Single 

1.0 

3,400 

30 

far 

Turbulent 

4. 

Single 

1.0 

3.400 

15 

far 

Turbulent 

6. 

Single 

1.0 

3,400 

30 

near 

Turbulent 

2. 

Single 

1.0 

3,400 

15 

near 

Turbulent 

4A  and  4B 

3. 

Single 

2.4 

MIHH 

15 

far 

Calm 

5. 

Single 

2.4 

30 

near 

Calm 

10. 

Fast/Slow 

2.4 

RKTWMHr 

15 

far 

Calm 

12. 

Distraction 

2.4 

30 

far 

Calm 

9. 

Fast/Slow 

2.4 

15 

near 

Calm 

5A  and  5B 

1. 

Single 

2.4 

3,400 

15 

near 

Calm 

13. 

Simult/Miss 

2.4 

3,400 

15 

0.5 

Calm 

7. 

Single 

2.4 

3,400 

30 

far 

Calm 

11. 

Distraction 

2.4 

3,400 

15 

far 

Calm 

6A  and  6B 

6. 

Single 

2.4 

3,400 

30 

near 

Turbulent 

4. 

Single 

2.4 

3,400 

15 

far 

Turbulent 

8. 

Single 

2.4 

3,400 

30 

far 

Turbulent 

2. 

Single 

2.4 

3,400 

15 

near 

Turbulent 

7A  and  7B 

5. 

Single 

4.8 

4,300 

30 

far 

Calm 

8. 

Fast/Slow 

4.8 

4,300 

15 

far 

Calm 

1. 

Single 

4.8 

4,300 

15 

near 

Calm 

9. 

Distraction 

4.8 

4,300 

15 

far 

Calm 

8A  and  8B 

3. 

Single 

4.8 

4,300 

30 

near 

Calm 

10. 

Distraction 

4.8 

4,300 

30 

far 

Calm 

11. 

SimuK/Miss 

4.8 

4,300 

15 

0.5 

Calm 

7. 

Fast/Slow 

4.8 

4,300 

15 

near 

Calm 

9A  and  9B 

4. 

Single 

4.8 

4,300 

30 

near 

Turbulent 

6. 

Single 

4.8 

4,300 

30 

far 

Turbulent 

2. 

Single 

4.8 

4,300 

15 

near 

Turbulent 
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APPENDIX  C 


Controller 

Pair 


Counterbalanced  Order 

of  Arrival 

Push 

Presentations 

1A 

2A 

3A 

4A 

5A 

6A 

7/ 

8A 

9A 

IB 

28 

3B 

4B 

5B 

68 

7 

88 

98 

2A 

3A 

4A 

5A 

6A 

7A 

8A 

9A 

1A 

2B 

38 

48 

SB 

6B 

78 

88 

9B 

18 

3A 

4A 

5A 

6A 

7A 

8A 

9A 

1A 

2A 

38 

48 

5B 

6B 

7B 

88 

98 

IB 

28 

4A 

5A 

6A 

7A 

8A 

9A 

1A 

2A 

3A 

48 

58 

6B 

78 

8B 

98 

IB 

28 

38 

5A 

6A 

7A 

8A 

9A 

1A 

2A 

3A 

4A 

58 

68 

78 

8B 

9B 

IB 

2B 

38 

48 

6A 

7A 

8A 

9A 

1A 

2A 

3A 

4A 

5A 

68 

78 

88 

98 

IB 

2B 

38 

48 

58 

7A 

8A 

9A 

1 A 

2A 

3A 

4A 

5A 

6A 

7B 

88 

98 

18 

2B 

38 

48 

58 

68 

8A 

9A 

1A 

2A 

3A 

4A 

5A 

6A 

7A 

88 

98 

IB 

2B 

3B 

48 

58 

68 

7B 

9A 

1A 

2A 

3A 

4A 

5A 

6A 

7A 

8A 

98 

IB 

2B 

3B 

4B 

58 

68 

78 

88 

1A 

2A 

3A 

4A 

5A 

6A 

7A 

8A 

9A 

18 

28 

3B 

48 

58 

68 

7B 

88 

98 

2A 

3A 

4A 

5A 

6A 

7A 

8A 

9A 

1A 

28 

38 

48 

58 

68 

78 

88 

98 

IB 

3A 

4A 

5A 

6A 

7A 

8A 

9A 

1A 

2A 

38 

48 

5B 

6B 

7B 

88 

98 

IB 

28 

4A 

5A 

6A 

7A 

8A 

9A 

1A 

2A 

3A 

48 

58 

6B 

7B 

SB 

98 

IB 

28 

38 

5A 

6A 

7A 

8A 

9A 

1A 

2A 

3A 

4A 

5B 

68 

78 

8B 

9B 

IB 

28 

38 

48 

6A 

7A 

8A 

9A 

1A 

2A 

3A 

4A 

5A 

68 

78 

88 

9B 

IB 

28 

38 

48 

58 

7A 

8A 

9A 

1A 

2A 

3A 

4A 

5A 

6A 

78 

88 

98 

IB 

2B 

38 

48 

58 

68 

8A 

9A 

1A 

2A 

3A 

4A 

5A 

6A 

7A 

88 

98 

IB 

2B 

3B 

48 

58 

68 

7B 

9A 

1A 

2A 

3A 

4A 

5A 

6A 

7A 

8A 

98 

18 

28 

3B 

48 

58 

68 

7B 

88 

1A 

2A 

3A 

4A 

5A 

6A 

7A 

8A 

9BA 

18 

28 

3B 

4B 

5B 

68 

7B 

88 

9 

2A 

3A 

4A 

5A 

6A 

7A 

8A 

9A 

1A 

28 

36 

4B 

5B 

6B 

7B 

88 

98 

IB 

3A 

4A 

5A 

6A 

7A 

8A 

9A 

1A 

2A 

38 

48 

5B 

6B 

7B 

88 

98 

IB 

28 

4A 

5A 

6A 

7A 

8A 

9A 

1A 

2A 

3A 

48 

58 

68 

7B 

SB 

96 

IB 

28 

38 

5A 

6A 

7A 

8A 

9A 

1A 

2A 

3A 

4A 

58 

68 

7B 

88 

98 

IB 

28 

38 

48 

6A 

7A 

8A 

9A 

1A 

2A 

3A 

4A 

5A 

68 

7B 

88 

9B 

IB 

2B 

38 

48 

58 

7A 

8A 

9A 

1 A 

2A 

3A 

4A 

5A 

6A 

7B 

88 

9B 

IB 

2B 

38 

4B 

58 

66 
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APPENDIX  D 


Arrival  Pushes  Shown  in  Study  II 


Note:  Study  II  used  a  subset  of  the  same  arrival  pushes  used  in  Study  I.  All  of  the  arrival  pushes  were 
presented  at  either  1.0-  or  2.4-s  sensor  update  interval.  The  difference  between  the  arrival 
pushes  used  in  Study  I  and  Study  II  is  that  the  runway  separation  in  Study  II  was  depicted  as  3,000 
ft  rather  than  the  previously  depicted  3,400  ft.  No  4.8-s  sensor  update  interval  approach  blunders 
were  used.  Only  calm  flight  path  conditions  were  used.  In  addition,  the  distraction  scenarios  were 
changed  to  be  single  blunders  without  distractions. 


Arrival  Push 

Blunder  Number 
and  Type 

Update 

Interval 

(sec) 

Runway 

Separation 

(ft) 

Angle 

(deg) 

Range 

(nml) 

Fit.  Path 
Conditions 

lAand  IB 

1.  Single 

1.0 

3,000 

15 

far 

Calm 

2.  Single 

1.0 

3,000 

30 

near 

Calm 

3.  Simu It/Miss 

1.0 

3,000 

15 

0.5 

Calm 

4.  Single 

1.0 

3,000 

30 

far 

Calm 

S.  Single 

1.0 

3,000 

30 

far 

Calm 

2A  and  2B 

6.  Past/Slow 

1.0 

3,000 

15 

far 

Calm 

7.  Single 

1.0 

3,000 

15 

far 

Calm 

8.  Single 

1.0 

3,000 

15 

near 

Calm 

9.  Fast/Slow 

1.0 

3,000 

15 

near 

Calm 

4A  and  4B 

14.  Single 

2.4 

3,000 

15 

far 

Calm 

15.  Single 

2.4 

3,000 

30 

near 

Calm 

16.  Fast/Slow 

2.4 

3,000 

15 

far 

Calm 

17.  Single 

2.4 

3,000 

30 

far 

Calm 

18.  Fast/Slow 

2.4 

3,000 

15 

near 

Calm 

5A  and  5B 

19.  Single 

2.4 

3,000 

15 

near 

Calm 

20.  Simuft/Miss 

2.4 

3,000 

15 

0.5 

Calm 

21.  Single 

2.4 

3,000 

30 

far 

Calm 

22.  Distraction 

2.4 

3,000 

15 

far 

Calm 
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APPENDIX  E 

Airline  and  Aircraft  Types  Included  in  the  Simulation 


Airline 

Percent  of  Flights 

NWA 

43 

DAL 

14 

NWX 

18 

FDX 

12 

USA 

6 

General  Aviation 

6 

UAL 

1 

Aircraft 

Percent  of  Flights 

B727 

31 

DC9 

20 

B757 

15 

BA31 

9 

SF34 

8 

B737 

7 

LR25 

2 

DC10 

2 

AC21 

2 

B767 

2 

BEnn 

0.6 

LR55 

0.6 

C172 

0.2 

PA30 

0.2 

C14 

0.2 

DH4 

0.2 
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APPENDIX  F 

PRECISION  RUNWAY  MONITOR 
CONTROLLER  OPINION  SURVEY 
FOR 

MEMPHIS  AND  RALEIGH-DURHAM  STUDIES 

Subject  ID: 
Interviewer  ID: 
Date: 


1. 

PROFESSIONAL  EXPERIENCE,  (to  the 

nearest  half-year) 

1.1 

Rada”  Control  Experience 

Years 

1.2 

Parallel  Runway  Monitor  Experience 

Years 

1.3 

Parallel  Runway  Monitor  E-system  Exposure 

□  Yes  Q  No  (RDU  ONLY) 

2.  GENERAL  ACCEPTANCE: 

2 . 1  PRM  enabled  me  to  monitor  the 

final  approach  better  than  the 
existing  ARTS  system. 


Comments: 

Agree _ 

Disagree _ 

Undecided 


2.2  PRM's  high  resolution  color  monitor  Agree _ 

is  better  for  the  monitor  function  Disagree _ 

than  the  current  ARTS  system  display.  Undecided  _ 


2 . 3  The  PRM  display  with  the  automated  alarms  made  it 
easier  to  detect  and  resolve  potential  and  actual 
blunders/deviations  better  than  the  existing 
ARTS  system. 

2.4  Independent  IFR  approaches  to  runways 
separated  by  3,400/3,500  ft  can  be 
safely  conducted  using  the  PRM. 


Agree _ 

Disagree  _ 
Undecided 


Agree _ 

Disagree  _ 
Undecided 


3.  MONITOR  CONTROLLER  FUNCTIONS 

3.1  PRM  is  useful  as  a  final  approach 

monitor  to  prevent  penetration  of 
the  NTZ. 


Commeots: 

Agree _ 

Disagree _ 

Undecided 


3.2  PRM  is  useful  in  resolving  approach 
blunders  once  they  have  occurred. 


Agree _ 

Disagree _ 

Undecided  _ 
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3.3 

PRM  is  useful  in  detecting  deviations 
from  the  designated  approach  course. 

Agree _ 

Disagree _ 

Undecided _ 

3.4 

PRM  is  useful  in  monitoring  simultaneous 
missed  approach  to  ensure  that  the  required 
divergence  is  achieved. 

Agree _ 

Disagree _ 

Undecided _ 

4. 

NTZ  ALERTS 

4.1 

The  Caution  Visual  Alert  (Yellow), 
predicting  x  seconds  or  less 
until  NTZ  penetration,  is  useful. 

Agree _ 

Disagree _ 

Undecided _ 

4.2 

The  Voice  Alert  accompanying  the 

Caution  Visual  Alert  (Yellow)  is 
useful. 

Agree _ 

Disagree _ 

Undecided _ 

4.3 

The  Warning  Visual  Alert  (Red), 
indicating  that  NTZ  penetration 
has  occurred,  is  useful. 

Agree _ 

Disagree _ 

Undecided _ 

5. 

DISPLAY  INFORMATION  CONTENT  AND 

PRESENTATION 

5.1 

The  information  presented  in  the  PRM 
display  is  well-placed  and  easily  vistole. 

Agree _ 

Disagree _ 

Undecided _ 

5.2 

The  written  information  presented  in  the  PRM 
display  is  easily  read. 

Agree _ 

Disagree _ 

Undecided _ 

5.3 

The  color  display  is  more  effective 
than  a  monochrome  display. 

Agree _ 

Disagree _ 

Undecided _ 

5.4 

The  ability  to  rotate  the  runways 
from  the  actual  runway  orientation 
to  either  the  vertical  or  horizontal 
is  a  sufficient  rotational 
capablity  (RDU  ONLY). 

Agree _ 

Disagree _ 

Undecided _ 

5.5 

The  parallel  200-ft  lines  are  a  useful  aid 
in  detecting  deviations  from  the  approach 
course,  and  in  predicting  the  potential  for 
an  NTZ  penetration. 

Agree _ 

Disagree _ 

Undecided _ 

5.6 

The  color  selection  for  the  features  on  the  display 
is  suitable. 

Agree _ 

Disagree _ 

Undecided _ 

Cammams: 


Comments: 
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6.  Instructions:  Listed  below  are  all  the 

information  items  automatically  provided  or 
available  by  selection.  Indicate  whether  you  agree, 
disagree,  or  are  undecided  that  the  item  is  useful 
in  assisting  you  to  perform  the  Monitor  Controller  task. 

List  any  changes  you  would  like  to  see  made 

in  the  information  included  (Content)  or  the  way  in  which  it  appears 

(Presentation). 


Hem  Content 


Presentation  Useful 


6.1  History  Trail 


6.2  Projected 
Position 
Vector 


6.3  200-ft 
deviation 
lines 


Any  aircraft  with  more  Green 

than  one  position  report  Dashes 

wifi  display  as  many  of 
them  as  requested. 


For  all  aircraft  that  Cyan 

have  at  least  two  good  Straight  line 

position  reports,  a 

trend  vector  is  drawn 

from  the  aircraft  symbol 

out  to  the  point  where  it 

would  be,  if  the  aircraft 

continued  at  its  current 

velocity, in  the  number 

of  seconds  you  have  selected. 


Indicate  200-ft  White  lines 

Increments  toward  NTZ 


Agree _ 

Disagree _ 

Undecided . 


Agree _ 

Disagree _ 

Undecided 


Agree _ 

Disagree _ 

Undecided 


7.  TRAINING 


7. 1  Adequate  training  time  was  provided  Agree 

to  become  familiar  with  the  display  Disagree _ 

before  beginning  the  Testing.  Undecided 

7.2  All  information  needed,  to  aid  me  in  Agree 

performing  the  monitoring  task,  Disagree _ 

was  provided.  Undecided 


(Please  comment  on  anything  omitted 
from  the  training  that  would  have 
helped  you  to  better  perform  the 
monitoring  task.) 


Comments 


Comments: 
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8.  THE  SIMULATION 

Comments 


8 . 1  The  simulated  traffic  density  was  Agree _ 

realistic.  Disagree _ 

Undecided  _ 

8.2  The  simulated  aircraft  "blunder"  Agree _ 

trajectories  were  realistic.  Disagree _ 

Undecided  _ 

8.3  The  simulated  aircraft  missed  Agree _ 

approach  trajectories  were  Disagree _ 

realistic.  Undecided . 

8.4  The  audio  portion  of  the  simulation  Agree _ 

was  realistic.  Disagree _ 

Undecided  _ 


8.5  Please  comment  on  any  changes  to 
the  simulation  which  you  feel  would 
enhance  its  realism. 

8.6  Were  there  any  factors  of  the 
simulation  which  may  have  affected 
the  quality  of  the  reaction  time 
measurement? 


9.  ADDITIONAL  COMMENTS: 
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APPENDIX  G 

PRECISION  RUNWAY  MONITOR 
CONTROLLER  OPINION  SURVEY 
FOR 

MEMPHIS  STUDY  II 

Subject  ID: 
Interviewer  ID: 
Date: 

1.  PROFESSIONAL  EXPERIENCE:  (to  the  nearest  half-year) 

1 . 1  Radar  Control  Experience  _ Years 

1 .2  Parallel  Runway  Monitor  Experience  _ Years 


2.  GENERAL  ACCEPTANCE: 

Comments: 

2.1  Independent  IFR  approaches  to  runways  Agree _ 

separated  by  3,000  ft  can  be  safely  Disagree _ 

conducted  using  the  PRM.  Undecided _ 


3.  TRAINING 

3 . 1  Adequate  training  time  was  provided 
to  become  familiar  with  the  display 
before  beginning  the  Testing. 

3.2  All  information  needed,  to  aid  me  in 
performing  the  monitoring  task, 
was  provided. 

(Please  comment  on  anything  omitted 
from  the  training  that  would  have 
helped  you  to  better  perform  the 
monitoring  task.) 


Comments- 

Agree _ 

Disagree _ 

Undecided _ 

Agree _ 

Disagree _ 

Undecided _ 


4.  THE  SIMULATION 

Cammeots 

4.1  The  simulated  traffic  density  was  Agree 

realistic.  Disagree _ 

Undecided _ 


4.2  The  simulated  aircraft  "blunder 
trajectories  were  realistic. 


Agree _ 

Disagree _ 

Undecided  _ 
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4.3  The  simulated  aircraft  missed 
approach  trajectories  were 
realistic. 

4.4  The  audio  portion  of  the  simulation 
was  realistic. 


4.5  Please  comment  on  any  changes  to 
the  simulation  which  you  feel  would 
enhance  its  realism. 

4.6  Were  there  any  factors  of  the 
simulation  which  may  have  affected 
the  quality  of  the  reaction  time 
measurement? 


Agree _ 

Disagree _ 

Undecided  _ 

Agree _ 

Disagree _ 

Undecided 


5.  ADDITIONAL  COMMENTS: 
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APPENDIX  H 


The  Blunder  Table 

Examples  of  Database  Queries  and  Replies 

The  following  is  a  listing  of  some  SQL  (Structure  Query  Language)  queries  and  replies.  After  logging  onto  the 
controller  response  database,  these  queries  may  be  executed  to  obtain  information  irom  the  on-line  ORACLE 
data  dictionary  concerning  the  structure  of  the  BLUNDER  table. 


Query  Example  «1:  This  query  returns  a  one  sentence  description  of  the  BLUNDER  table. 

SQL  >  SELECT  COMMENTS 

2  FROM  SYS.ALL_TAB_COMMENTS 

3  WHERE  TABLE_NAME  « 'BLUNDER* 

4  AND  OWNER  =  7EUS'; 

HSAlX.: 

BLUNDER:  This  table  contains  the  parameters  of  each  scripted  blunder  used  in  the  controller  response 

study. 


Query  Example  #2:  This  query  returns  the  names  of  the  columns  (blunder  parameters)  in  the  BLUNDER 

table,  whether  the  column  must  contain  data  for  each  row  (NOT  NULL),  and  the  data 
type  of  each  column.  NUMBER(38)  is  the  ORACLE  format  for  an  integer. 
NUMBER(2,1)  is  the  ORACLE  format  for  a  two  digit  number  with  one  digits  to  the  right 
of  the  decimal  point.  CHAR(8)  is  the  ORACLE  format  for  a  character  array  containing  at 
most  eight  characters. 

SQL  >DESC  BLUNDER; 

Reply: 


Mans 

NuB? 

Ix&e 

SCENARIO 

NOT  NULL 

CHAR(8) 

MAP 

CHAR(8) 

BLUNDER 

NOT  NULL 

CHAR(4) 

TYPE 

NOT  NULL 

CHAR(24) 

ANGLE 

NOT  NULL 

NUMBER(38) 

RANGE 

NOT  NULL 

NUMBER(38) 

PERIOD 

NOT  NULL 

NUMBER(2,1) 

WEATHER 

NOT  NULL 

CHAR(16) 

SEAT 

NOT  NULL 

CHAR(8) 

Query  Example  #3:  This  query  returns  a  more  detailed  description  of  the  ranges  and  representative  values 

of  each  column  in  the  BLUNDER  table. 

SQL  >  SELECT  COMMENTS 

2  FROM  SYS.ALL_COL_COMMENTS 

3  WHERE  TABLE_NAME  -  ’BLUNDER’ 

4  AND  OWNER  -  7EUS'; 


The  simulated  blunder  scenario  tape  identifier;  e.g.,  VII,  IX-A. 

The  PRM  display  map  selected  on  the  monitor,  NORTH  or  SOUTH. 

The  unique  blunder  identifier  from  1  to  37. 

The  type  of  blunder;  e.g.,  SINGLE,  DISTRACTION,  FAST/SLOW,  SIMULTANEOUS/MISS. 
The  blunder  angle  in  degrees  from  the  ILS,  15  or  30. 

The  distance  from  the  threshold  of  the  blunder  (nmi);  e.g.,  0.5, 2, 3, 4, 8, 9,10, 11,12. 

The  display  update  period  in  seconds;  e.g.,  0.5, 1.0,  2.4,  4.8. 

The  weather  conditions  during  a  parallel  approach;  e.g.  CALM,  TURBULENT. 

The  controller  monitor  seat;  e.g.,  LEFT,  RIGHT. 


Reply: 

SCENARIO: 

MAP: 

BLUNDER: 

TYPE: 

ANGLE: 

RANGE: 

PERIOD: 

WEATHER: 

SEAT: 


APPENDIX  I 


Controller  Response  Table 
Examples  of  Database  Queries  and  Replies 

The  following  is  a  listing  of  some  SQL  (Structure  Query  Language)  queries  and  replies.  After  logging  onto  the 
controller  response  database,  these  queries  may  be  executed  to  obtain  information  from  the  on-line  ORACLE 
data  dictionary  concerning  the  structure  of  the  CONTROLLER_RESPONSE  table. 


Query  Example  #1:  This  query  returns  a  one  sentence  description  of  the  CONTROLLER_RESPONSE 

table. 

SQL  >  SELECT  COMMENTS 

2  FROM  SYS.ALL_TAB_COMMENTS 

3  WHERE  TABLE_NAME  - 'CONTROLLER  RESPONSE' 

4  AND  OWNER  *  'ZEUS'; 

ELep.ly; 

CAUTION_ALERT :  This  table  contains  the  times  from  onset  of  the  yellow  alert  to  controller  response. 


Querv  Example  *2:  This  query  returns  the  names  of  the  columns  in  the  CONTROLLER  RESPONSE 

table,  whether  the  column  must  contain  data  for  each  row  (NOT  NULL),  and  the  data 

type  of 

each  column. 

NUMBER(38)  is  the  ORACLE  format  for  an  integer. 

NUMBER(4,2)  is  the  ORACLE  format  for  a  four  digit  number  with  two  of  the  four  digits 

to  the  right  of  the  decimal  point.  CHAR(4)  is  the  ORACLE  format  for  character  array  of 

at  most  four  characters. 

SQL  >  DESC  CONTROLLER_RESPONSE; 

Reply: 

Name 

Nun? 

Type 

WEEK 

NUMBER(38) 

BLUNDER 

NOT  NULL 

CHAR(4) 

AIRPORT 

NOT  NULL 

CHAR(4) 

CONTROLLER 

NOT  NULL 

CHAR(8) 

RESPTIME 

NUMBER(4,2) 

SEAT 

NOT  NULL 

CHAR(8) 

STAMP 

NOT  NULL 

CHAR(24) 

Query  Example.  J£3: 


This  query  returns  a  more  detailed  description  of  the  ranges  and  representative 
values  for  each  column  in  the  CONTROLLER  RESPONSE  table. 


SQL  >  SELECT  COMMENTS 

2  FROM  SYS.ALL_COL_COMMENTS 

3  WHERE  TABLE_NAME  «  'CONTROLLER_RESPONSE' 

4  AND  OWNER  «*ZEUS'; 


Reply: 

WEEK: 

BLUNDER: 

AIRPORT: 

CONTROLLER: 

RESPTIME: 

SEAT: 

STAMP: 


The  number  of  the  week  of  the  study  for  this  measurement,  1  through  25. 
The  unique  blunder  identifier  from  1  to  29,  30L,  30R,  31L,  31R  and  32  to  37. 
The  three  letter  airport  designation;  MEM,  THB,  LXR. 

The  test  subject  (controller)  identifier;  e.g.,  M3-B,  G12-A. 

The  controller  response  time,  In  seconds,  to  the  warning  of  a  blunder. 

The  controller  monitor  seat;  e.g.,  LEFT,  RIGHT. 

A  quality  control  stamp  or  description  of  the  measurement. 
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APPENDIX  J 


The  Caution  Atert  Table 
Examples  of  Database  Queries  and  Replies 


The  following  is  a  listing  of  some  SQL  (Structure  Query  Language)  queries  and  replies.  After  logging  onto  the 
controller  response  database,  these  queries  may  be  executed  to  obtain  information  from  the  on-line  ORACLE 
data  dictionary  concerning  the  structure  of  the  BLUNDER  table. 


Query  Example  #1:  This  query  returns  a  one  sentence  description  of  the  CAUTION_ALERT  table. 

SQL  >  SELECT  COMMENTS 

2  FROM  SYS.ALL_TAB_COMMENTS 

3  WHERE  TABLE_NAME  -  'C AUT ION_AL E RT 

4  AND  OWNER  -  ZEUS’; 

Beoly: 

CAUT10N_ALERT :  This  table  contains  the  caution  alert  lead  times  for  each  blunder  from  the  controller  study. 


Query  Example  #2:  This  query  returns  the  names  of  the  columns  in  the  CAUTION_ALERT  table,  whether 
the  column  must  contain  data  for  each  row  (NOT  NULL),  and  the  data  type  of  each 
column.  NUMBER(38)  is  the  ORACLE  format  for  an  integer.  NUMBER(4,2)  is  the 
ORACLE  format  for  a  four  digit  number  with  two  of  the  four  digits  to  the  right  of  the 
decimal  point.  CHAR(8)  is  the  ORACLE  format  for  a  character  array  containing  at  most 
eight  characters. 


SQL  >  DESC  C AUTION_ALERT ; 

Name 

Null? 

Typa 

SCENARIO 

NOT  NULL 

CHAR(8) 

BLUNDER 

NOT  NULL 

CHAR(4) 

AIRPORT 

NOT  NULL 

CHAR(4) 

CONTROLLER 

NOTNULL 

CHAR(8) 

LEADTIME 

NUMBER(4,2) 

SEAT 

NOT  NULL 

CHAR(48) 

Query  Example  #3:  This  query  returns  a  more  detailed  description  of  the  ranges  and  representative  values  of 
each  column  in  the  BLUNDER  table. 

SQL  >  SELECT  COMMENTS 

2  FROM  SYS. ALL  COL_COMMENTS 

3  WHERE  TABLE_NAME  -  'CAUTION_ALERT 

4  AND  OWNER  -  ZEUS’; 

Reply: 

SCENARIO;  The  simulated  blunder  scenario  tape  identifier;  e.g.  ,VII,  IX-A. 

BLUNDER:  The  unique  blunder  identifier  from  1  to  29, 30L,  30R,  31 L,  31 R  and  32  to  37. 

AIRPORT:  The  three  letter  airport  designation;  MEM,  THB,  LXR. 

CONTROLLER:  The  test  subject  (controller)  identifier;  e.g.,  M3-B,  G12-A. 

LEADTIME:  The  data  cluster  average  of  the  measured  caution  alert  lead  time  (sec). 

STAMP:  A  quality  control  stamp  or  description  of  the  measurement. 
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APPENDIX  K 


Adjusted  Probability  Criteria  for  Wlthln-Subject  Comparisons 
Performed  In  Study  I  and  II 


Study  1  •  Section 

Analysis  Area 

Number  of  Pisnned 
Within-Subject 
Comparisons 

Probability  Criteria 
(Bonferroni 
Adfuated) 

4.1.1  thru  4.1.6 

1.0-s  and  2.4-s  sensor  update  interval, 
3,400-ft  runway  separation 

23 

4.2.1  and  4.2.2 

4.S-s  sensor  update  interval,  3,400-ft 
runway  separation 

3 

4.4.1  thru  4.4.4 

4.8-s  sensor  update  interval,  4,300-ft 
runway  separation 

4 

Total:  30 

.001  or  less 

Study  II  -  Section 

Analysis  Area 

Number  of  Planned 
Within-Subject 
Comparisons 

Probability  Criteria 
(Bonferroni 
Adlusted) 

5.2  thru  5.4 

1.?  ;  anu  2.4  s  ser.sor  update  interval, 
3,000-ft  runway  separation 

10 

.005  or  less 
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